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Abstract 

Given a short exact sequence of groups with certain conditions, 1 F — >■ G H — ► 1, we 
prove that G has solvable conjugacy problem if and only if the corresponding action subgroup 
A $C Aut(F) is orbit decidable. From this, we deduce that the conjugacy problem is solvable, 
among others, for all groups of the form Z 2 x F m , Fi x F m , F n x Z, and Z n x^ F m with virtually 
solvable action group A ^ GL n (Z). Also, we give an easy way of constructing groups of the 
form Z 4 x F n and F3 x F n with unsolvable conjugacy problem. On the way, we solve the twisted 
conjugacy problem for virtually surface and virtually polycyclic groups, and give an example 
of a group with solvable conjugacy problem but unsolvable twisted conjugacy problem. As an 
application, an alternative solution to the conjugacy problem in Aut(F2) is given. 



1 Introduction 

Let G be a group, and u,v £ G. The symbol ~ will be used to denote standard conjugacy in G 
(u ~ v if there exists x S G such that v = x~ l ux). In this paper, we shall work with a twisted 
version, which is another equivalence relation on G. Given an automorphism ip £ Aut(G), we say 
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that u and v are (p-twisted conjugated, denoted u v, if there exists x G G such that v = (x(p) ux. 
Of course, coincides with ~. Reidemeister was the first author considering the relation ~„ 
(see [35J), which has an important role in modern Nielsen fixed point theory. A few interesting 
references can be found in [T7], where it is proven that the number of 92-twisted conjugacy classes 
in a non-elementary word-hyperbolic group is always infinite; [6], where an algorithm is given 
for recognizing (^-twisted conjugacy classes in free groups; and [18j . where the notion of twisted 
conjugacy separability is analyzed. 

Precisely, the recognition of twisted conjugacy classes is one of the main problems focused on in 
the present paper. The twisted conjugacy problem for a group G consists on finding an algorithm 
which, given an automorphism ip S Aut(G) and two elements u,v € G, decides whether v ~<~ u or 
not. Of course, a positive solution to the twisted conjugacy problem automatically gives a solution 
to the (standard) conjugacy problem, which in turn provides a solution to the word problem. 
The existence of a group G with solvable word problem but unsolvable conjugacy problem is well 
known (see [30] ) . In this direction, one of the results here is the existence of a group with solvable 
conjugacy problem, but unsolvable twisted conjugacy problem (see Corollary 14.91 below). 

Let us mention the real motivation and origin of the present work. Several months ago, the 
same three authors (together with O. Maslakova) wrote the paper [BJ, where they solved the twisted 
conjugacy problem for finitely generated free groups and, as a corollary, a solution to the conjugacy 
problem for free-by-cyclic groups was also obtained. A key ingredient in this second result was 
Brinkmann's theorem, saying that there is an algorithm such that, given an automorphism a of 
a finitely generated free group and two elements u and v, decides whether v is conjugate to some 
iterated image of u, v ~ ua k (see [1Q]). At some point, Susan Hermiller brought to our attention 
an article due to Miller, [30], where he constructed a free-by-free group with unsolvable conjugacy 
problem. It turns out that the full proof in [6J extends perfectly well to the bigger family of 
free-by-free groups without any problem, except for a single step in the argumentation: at the 
point in [6] where we used Brinkmann's result, a much stronger problem arises, which we call orbit 
decidability (see below for definitions). This way, orbit decidability is really the unique obstruction 
when extending the arguments from free-by-cyclic to free-by-free groups. 

Hence, we can deduce that a free-by-free group has solvable conjugacy problem if and only 
if the corresponding orbit decidability problem is solvable. This idea is formally expressed in 
Theorem 13. H the main result of the present paper. In fact, Theorem 13.11 works not only for free-by- 
free groups, but for an even bigger family of groups. And, of course, when we restrict ourselves to 
the free-by-cyclic case, orbit decidability becomes exactly the algorithmic problem already solved 
by Brinkmann's theorem. 

More precisely, Theorem 13.11 talks about a short exact sequence 1 ^ i 7 — > G — > H — > 1 with 
some conditions on F and H. And states that the group G has solvable conjugacy problem if and 
only if the action subgroup Aq ^ Aut{F) is orbit decidable. Here, Aq is the group of elements 
in G acting by right conjugation on F; and being orbit decidable means that, given u,v £ F, 
we can algorithmically decide whether v ~ ua for some a G Aq. Of course, if we take F to be 
free and H to be infinite cyclic, we get free-by-cyclic groups for G; and the action subgroup being 
orbit decidable is precisely Brinkmann's theorem (see Subsection I6.2|) . The conditions imposed to 
the groups F and H are the twisted conjugacy problem for F, and the conjugacy problem plus a 
technical condition about centralizers for H . 

In light of Theorem 13.11 it becomes interesting, first, to collect groups F where the twisted 
conjugacy problem can be solved. And then, for every such group F, to study the property 
of orbit decidability for subgroups of Aut(F): every orbit decidable (undecidable) subgroup of 
Aut(F) will correspond to extensions of F having solvable (unsolvable) conjugacy problem. After 
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proving Theorem 13.11 this is the main goal of the paper. We first show how to solve the twisted 
conjugacy problem for surface and polycyclic groups (in fact, the solution of these problems follows 
easily from the solution of the conjugacy problem for polycylic and for fundamental groups of 3- 
manifolds, respectively). We also prove that the solvability of this problem goes up to finite index 
for these classes of groups. Then, we concentrate on the simplest examples of groups with solvable 
twisted conjugacy problem, namely finitely generated free (say F n ) and free abelian (Z ra ). On the 
positive side, we find several orbit decidable subgroups of Aut(F n ) and Aut(7 J n ) = GL n (Z), with 
the corresponding free- by- free and free abelian- by- free groups with solvable conjugacy problem. 
Probably the most interesting result in this direction is Proposition l6.9l saying that virtually solvable 
subgroups of GL n (Z) are orbit decidable; or, say in a different way, orbit undecidable subgroups 
of GL n (]L) must contain F%. On the negative side, we establish a source of orbit undecidability, 
certainly related with a special role of F% inside the automorphism group. In particular, this will 
give orbit undecidable subgroups in Aut(F n ) for n ^ 3, which will correspond to Miller's free-by-free 
groups with unsolvable conjugacy problem; and orbit undecidable subgroups in GL n (Z) for n ^ 4, 
which will correspond to the first known examples of Z 4 -by-free groups with unsolvable conjugacy 
problem. 

All over the paper, F n denotes the free group of finite rank n ^ 0; and F, G and H stand for 
arbitrary groups. Having Theorem 13.11 in mind, we will use one or another of these letters to make 
clear which position in the short exact sequence the reader should think about. For example, typical 
groups to put in the first position of the sequence are free groups (F n ), and in the third position are 
hyperbolic groups (H). We will write morphisms as acting on the right, x i— > x<p. In particular, the 
inner automorphism given by right conjugation by g £ G is denoted ^ g : G —* G, x i— > xj g = g~ x xg. 
As usual, End{G) denotes the monoid of endomorphisms of G, Aut{G) the group of automorphisms 
of G, Inn(G) denotes the group of inner automorphisms, and Out{G) = Aut(G) / Inn{G). Finally, 
we write [u,v] = u~ l v~ 1 uv, and Cg{u) = {g £ G \ gu = ug} ^ G for the centralizer of an element 
u. 

As usual, given a group F = (X \ R) and m automorphisms (pi, . . . ,<p m € Aut(F), the free 
extension of F by tpi , . . . , ip m is the group 

F >V,-,¥>m F m = (X,h, ...,t m \R, t~ l xti = xipi (x e X, i = 1, . . . ,m)). 

Such a group is called F-by-[f.g. free]. In particular, if m = 1 we call it F -by-cyclic and denote 
by F xi pi Z. It is well known that a group G is F-by-ff.g. free] if and only if it has a normal 
subgroup isomorphic to F, with finitely generated free quotient G/F (i.e. if and only if it is the 
middle term in a short exact sequence of the form 1— > F ^ G ^ F n ^ 1). Above, when talking 
about "free-by-free" and "free abelian-by-free" we meant [f.g. free] -by- ff.g. free] and [f.g. abelian] - 
by-[f.g. free], respectively. To avoid confusions, we shall keep these last names; they make an 
appropriate reference to finite generation, and they stress the fact that parenthesis are relevant 
(note, for example, that surface groups are both free-by-cyclic and finitely generated, but most of 
them are not [f.g. free]-by-cyclic). 

The paper is structured as follows. In Section [2] we discuss some preliminaries concerning 
algorithmic issues, setting up the notation and names used later. Section [3] contains the first 
part of the work, developing the relation between the conjugacy problem and the concept of orbit 
decidability. In Section 0] the applicability of Theorem 13. II is enlarged, by finding more groups with 
solvable twisted conjugacy problem (Subsection 14. ip . and by proving that many hyperbolic groups 
have small centralizers in an algorithmic sense (Subsection l4.2|) . Then, Section[5]is dedicated to solve 
the conjugacy problem for Aut(F2)\ this problem was already known to be solvable (see [3] and [1]), 
but we present here an alternative solution to illustrate an application of the techniques developed 
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in the paper. Sections [6] and [7J are dedicated, respectively, to several positive and negative results, 
namely orbit decidable subgroups corresponding to extensions with solvable conjugacy problem, 
and orbit undecidable subgroups corresponding to extensions with unsolvable conjugacy problem. 
Both, in the free abelian case, and in the free case. Finally, Section [8] is dedicated to summarize 
and comment on some questions and open problems. 

2 Algorithmic preliminaries 

From the algorithmic point of view, the sentence "let G be a group" is not precise enough: the 
algorithmic behaviour of G may depend on how G is given to us. For the purposes of this paper, we 
assume that a group will always be given to us in an algorithmic way: elements must be represented 
by finite objects and multiplication and inversion must be doable in an algorithmic fashion; also, 
morphisms between groups, a : F — > G, are to be represented by a finite amount of information in 
such a way that one can algorithmically compute images of elements in F. 

If G is finitely presented, a natural way (though not the unique one) consists in giving the 
group G by a finite presentation, G = {x±, . . . , x n | ri, . . . , r m ); here, elements of G are represented 
by words in the Xj's, multiplication and inversion are the standard ones in the free group (modulo 
relations), and morphisms are given by images of generators. 

Let G and <f>: G — > G be a group and an automorphism given in an algorithmic way. Also, let 
F ^ G be a subgroup. The following are interesting algorithmic problems in group theory (the 
first two typically known as Dehn problems): 

• the word problem for G, denoted WP(G): given an element w G G, decide whether it 
represents the trivial element of G. It is well known that there exists finitely presented 
groups with unsolvable word problem (see [31] or [7]). 

• the conjugacy problem for G, denoted CP(G): given two elements u,v G G, decide whether 
they are conjugate to each other in G. Clearly, a solution for the CP(G) immediately gives a 
solution for the WP(G), and it is well known the existence of finitely presented groups with 
solvable word problem but unsolvable conjugacy problem (see, for example, [11] or |30j). 

• the ^-twisted conjugacy problem for G, denoted TCP^(G): given two elements u,v G G, 
decide whether they are ^-twisted conjugate to each other in G (i.e. whether v = (gcj))~ 1 ug 
for some g G G). Note that TCP H (G) is CP(G). 

• the (uniform) twisted conjugacy problem for G, denoted TCP(G): given an automor- 
phism 4> G Aut(G) and two elements u,v G G, decide whether they are ^-twisted conjugate 
to each other in G. This is part of a more general problem posted by G. Makanin in Ques- 
tion 10.26(a) of [23]. Obviously, a solution for the TCP(G) immediately gives a solution for 
the TCP^G) for all <\> G Aut(G) (in particular, CP(G) and WP(G)). In section El we 
give an example of a finitely presented group with solvable conjugacy problem but unsolvable 
twisted conjugacy problem. 

• the membership problem for F in G, denoted MP(F, G): given an element w G G decide 
whether it belongs to F or not. There are well-known pairs (F, G) with unsolvable MP(i ? , G) 
(see [29] or [30]). 
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Conjugacy and twisted conjugacy problems have the "search" variants, respectively called the 
conjugacy search problem, CSP(G), and the twisted conjugacy search problem, TCSP(G), 
for G. They consist on additionally finding a conjugator (or twisted-conjugator) in case it exists. 

When groups are given by finite presentations and morphisms by images of generators, the 
"yes" parts of the listed problems are always solvable. Let G = (xi,...,x n \ ri,...,r m ) and 
H = (yi , . . . , y n < | si, . . . , s m i) be two finitely presented groups, (ft: G — > H be a morphism (given 
by xi i— ► Wi (j/i,..., Dn' ) i i = 1) ■ ■■■>n), and let F ^ G be a subgroup given by a finite set of 
generators . . . , x n ), . . . , ft(xi, . . . , x n )}. We have: 

• the "yes" part of the word problem for G, denoted WP + (G): given an element of G as 
a (reduced) word on the generators, w(x±, . . . ,x n ) G G, which is known to be trivial in G, 
find an expression of w as a product of conjugates of the relations r^. No matter if WP(G) 
is solvable or not, WP + (G) is always solvable by brute force: enumerate the normal closure 
R = ((n, . . . , r m }) in the free group (x±, . . . , x n | ), and check one by one whether its elements 
equal w as a word after reduction; since we know that w =q 1, the process will eventually 
terminate. Note that, without the assumption, this is not an algorithm because if w 1 
then it works forever without giving any answer. 

• the "yes" part of the conjugacy problem for G, denoted CP + (G): given two elements 
u(xi, . . . ,x n ), v(xi, . . . ,x n ) G G which are known to be conjugate to each other in G, find 
w(xi, . . . , Xn) such that w~ 1 uw =g v. Again, no matter if CP(G) or even WP(G) are solvable 
or not, CP + (G) is always solvable by brute force: enumerate the elements in the free group 
F = (xi, . . . ,x n | ) and for each one w(xi, . . . ,x n ) apply WP + (G) to v~ 1 w~ 1 uw . We know 
that if v ~ l w~ 1 uw 1 this process will never terminate. But we can start them for several 
w's and, while running, keep opening similar processes for new w's; eventually, one of them will 
stop telling us which w satisfies w~ 1 uw =q v. As before, note that, without the assumption, 
this is not an algorithm because if u and v are not conjugate to each other in G then it works 
forever without giving any answer. 

• the "yes" part of the twisted conjugacy problem for G, denoted TCP + (G): it is defined, 
and solved by brute force, in the exact same way as CP + (G). 

• the "yes" part of the membership problem for F in G, denoted MP + (F, G): given an 
element w(x\, . . . ,x n ) G G known to belong to F, express w as a word on the /j's. In a 
similar way as above, even if MP(F, G) is unsolvable, MP + (F, G) is always solvable by brute 
force. 

Note that, using these brute force arguments, the conjugacy problem is solvable if and only if the 
conjugacy search problem is solvable. Similarly, the twisted conjugacy problem is solvable if and 
only if the twisted conjugacy search problem is solvable. However, the corresponding complexities 
may be rather different. 

Finally, let us state few more problems, which will be considered in the present paper. We have: 

• the coset intersection problem for G, denoted CIP(G): given two finite sets of elements 
{ai,...,a r } and {bi,...,b s } in G, and two elements x,y G G decide whether the coset 
intersection xA n yB is empty or not, where A = {a±, . . . , a r ) ^ G and B = (b±, . . . , b s ) ^ G. 
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Let F < G be a normal subgroup of G. By normality, two elements u,v £ G conjugate to each 
other in G, either both belong to F or both outside F. Accordingly CP(G) splits into two parts, 
one of them relevant for our purposes: 

• the conjugacy problem for G restricted to F, denoted CPi?(G): given two elements 
u,v € F, decide whether they are conjugate to each other in G. Note that a solution to 
CP(G) automatically gives a solution to CPp(G), and that this is not, in general, the same 
problem as CP(F). 

Now, let F be a group given in an algorithmic way, and A be a subgroup of Aut(F). We have: 

• the orbit decidability problem for A, denoted OT)(A): given two elements u,v £ F, decide 
whether there exists (p 6 A such that uip and v are conjugate to each other in F. As will be 
seen in Section^ there are finitely generated subgroups A ^ Aut(F) with unsolvable OD(A). 

Note that orbit decidability for A ^ Aut(F) is equivalent to the existence of an algorithm which, 
for any two elements u, v G F, decides whether there exists ip £ A ■ Inn(F) such that u<p = v. 
In particular, if two subgroups A,B ^ Aut(F) satisfy A ■ Inn(F) = B ■ Inn(F) then OD(j4) and 
OD(B) are the same problem (in particular, A is orbit decidable if and only if B is orbit decidable). 
This means that orbit decidability is, in fact, a property of subgroups of Out(F). However, we 
shall keep talking about Aut(F) for notational convenience. 

To finish the section, let us consider an arbitrary short exact sequence of groups, 

1 — ► F -^4 G H — ► 1. 

Such a sequence is said to be algorithmic if: (i) the groups F, G and H and the morphisms a and (5 
are given to us in an algorithmic way, i.e. we can effectively operate in F, G and H, and compute 
images under a and /3, (ii) we can compute at least one pre-image in G of any element in H, and 
(iii) we can compute pre-images in F of elements in G mapping to the trivial element in H. 

The typical example (though not the unique one) of an algorithmic short exact sequence is 
that given by finite presentations and images of generators. In fact, (i) is immediate, and we can 
use MP + (G/3, H) to compute pre-images in G of elements in H, and use MP + (i ? a, G) to compute 
pre-images in F of elements in G mapping to 1h- 

3 Orbit decidability and the conjugacy problem 

Consider a short exact sequence of groups, 

1 — ► F -^4 G H — >1. 

Since Fa is normal in G, for every g E G, the conjugation j g of G induces an automorphism of 
F, x i— > g~ 1 xg, which will be denoted ip g € Aut(F) (note that, in general, (p g does not belong to 
Inn(F)). It is clear that the set of all such automorphisms, 

A G = {<p g \ge G}, 

is a subgroup of Aut(F) containing Inn(F). We shall refer to it as the action subgroup of the given 
short exact sequence. 

Assuming some hypothesis on the sequence and the groups involved on it, the following theorem 
shows that the solvability of the conjugacy problem for G is equivalent to the orbit decidability of 
the action subgroup Aq ^ Aut(F). 
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Theorem 3.1 Let 

1 — ► F -2U G H — >1. 
be an algorithmic short exact sequence of groups such that 
(i) F has solvable twisted conjugacy problem, 
(ii) H has solvable conjugacy problem, and 

(Hi) for every 1 ^ h G H , the subgroup (h) has finite index in its centralizer Cu{h), and there is 
an algorithm which computes a finite set of coset representatives, Zh,\, • • • , Zh,t h 6 H, 

C H {h) = (h)z h>x U---U{h)z h ,t h . 

Then, the following are equivalent: 

(a) the conjugacy problem for G is solvable, 

(b) the conjugacy problem for G restricted to F is solvable, 

(c) the action subgroup Aq = {ip g \ g G G} Aut(F) is orbit decidable. 

Proof. As usual, we shall identify F with Fa ^ G. By definition, xtp g = g~ 1 xg for every g € G and 
x G F. So, given two elements x, x' G F, finding g G G such that x' = g~ l xg is the same as finding 
ip G Aq such that x' = xip. Since Ac = Aq ■ Inn(F), this is solving OD(Ag)- Hence, (6) and (c) 
are equivalent. It is also obvious that (a) implies (6). So, the relevant implication is (6) =4> (a). 

Assume that (b) holds, let g,g' G G be two given elements in G and let us decide whether they 
are conjugate to each other in G. 

Map them to H. Using (ii), we can decide whether g/3 and g'/3 are conjugate to each other in 
H. If they are not, then g and g' cannot either be conjugate to each other in G. Otherwise, (ii) 
gives us an element of H conjugating g/3 into g'/3. Compute a pre-image u G G of this element. It 
satisfies {g u )f3 = (g/3) u P = g'(3- Now, changing g to g u , we may assume that g/3 = g'(3. If this is 
the trivial element in H (which we can decide because of (ii)) then g and g' lie in the image of a, 
and applying (b) we are done. Hence, we are restricted to the case g/3 = g'/3 1- 

Now, compute / G F such that g' = gf (this is the a-pre-image of g~~ 1 g')- Since g/3 j^h 1> we 
can use (iii) to compute elements z%, . . . , Zt G H such that Cuigfi) = (g/3)zi U • • • U {g(3)z t , and then 
compute a pre-image y, L G G for each Zi, i = 1, . . . , t. Note that, by construction, the /3-images of 
g and yi (respectively g/3 and Zi) commute in H so, y~ X gy% = gp% for some computable Pi G F. 

Since g/3 = g'/3, every possible conjugator of g into g' must map to CnigP) under (3 so, it must 
be of the form g r yiX for some integer r, some i G {1, . . . , t}, and some x G F. Hence, 

gf = g' = {x' 1 y^ 1 g' r )g(g r y i x) = x~ l {yl l gyi)x = x~ x gpiX. 

Thus, deciding whether g and g' are conjugate to each other in G amounts to decide whether there 
exists i G {1, . . . ,t} and x G F satisfying gf = x~ x gpiX, which is equivalent to / = {g~ l x~ l g)p; L x 
and so to / = {xtp g )~ l piX. Since i takes finitely many values and the previous equation means 
precisely / ~ v , 9 Pi, we can algorithmically solve this problem by hypothesis (i). This completes the 
proof. □ 

Remark 3.2 Note that in the proof of Theorem l3.1l we did not use the full power of hypothesis (i). 
In fact, we used a solution to TCP^(-F) only for the automorphisms in the action subgroup, (j) G Aq. 
For specific examples, this may be a weaker assumption than a full solution to TCP(F). 
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Theorem 13. 1 1 gives us a relatively big family of groups G for which the conjugacy problem reduces 
to its restriction to a certain normal subgroup. Now, we point out some examples of groups F and 
H satisfying hypotheses (i)-(iii) of the Theorem, and hence providing a family of groups G for 
which the characterization is valid. 

Suppose that F is a finitely generated abelian group. For any given <j> £ Aut(F) and u,v £ F, 
we have u v if and only if u = v modulo Im{4> — Id). Hence, TCP(i ? ) reduces to solving a 
system of linear equations (some over Z and some modulo certain integers). So it is solvable. On 
the other hand, Theorem 1.5 in [6] states that TCP(-F) is also solvable when F is finitely generated 
free. So, finitely generated abelian groups, and finitely generated free groups satisfy (i). 

With respect to conditions (ii) and (iii), it is well known that in a free group H, the centralizer 
of an element 1 ^ h £ H is cyclic and generated by the root of h (i.e., the unique non proper power 
h £ H such that h is a positive power of h), which is computable. Clearly, then, finitely generated 
free groups satisfy hypotheses (ii) and (iii). 

Focusing on these families of groups, Theorem 13.11 is talking about solvability of the conju- 
gacy problem for [f.g. free] -by- [f.g. free] and [f.g. abelian]-by-[f.g. free] groups, and can be restated 
as follows. 

Theorem 3.3 Let F = (x%, . . . ,x n \ R) be a (finitely generated) free or abelian group, and let 
tpi,...,(p m £ Aut(F). Then, the [f.g. free] -by- [f.g. free] or [f.g. abelian] -by- [f.g. free] group 

G — (xi , . . . , x n , t\ , . . . , t m | R, tj Xitj — x^ifj , i — 1, . . . , n, j — 1, . . . , m) 

has solvable conjugacy problem if and only if {ipi, . . . , (p m ) ^ Aut(F) is orbit decidable. □ 

In the following section, applicability of Theorem 13.11 will be enlarged by finding more groups 
with solvable twisted conjugacy problem (see Subsection l4.ip . and finding more groups which satisfy 
condition (iii) (see Subsection I4.2p . 

At this point, we want to remark that the study of the conjugacy problem in the families of 
[f.g. free] -by- [f.g. free] and [f.g. free abelian]-by-[f.g. free] groups was the authors' original motivation 
for developing the present research. One can interpret Theorem l3.3l bv saying that orbit decidability 
is the unique possible obstruction to the solvability of the conjugacy problem within these families 
of groups. So, by finding examples of orbit decidable subgroups in Aut(F n ) or Aut(Z n ) = GL n (7j) 
one is in fact solving the conjugacy problem in some [f.g. free]-by- [f.g. free] or [f.g. free abelian]-by- 
[f.g. free] groups; we develop this in Section [6j 

But in the literature there are known examples of [f.g. free]-by-[f.g. free] groups with unsolvable 
conjugacy problem: Miller's example of such a group (see Miller [30] ) automatically gives us an 
example of a finitely generated subgroup of the automorphism group of a free group, which is 
orbit undecidable. We discuss this in Section [TJ where we also find orbit undecidable subgroups in 
GL n (]L) for n ^ 4; these last examples correspond to the first known [f.g. free abelian]-by- [f.g. free] 
groups with unsolvable conjugacy problem. 

To conclude the section, we point out the following consequence of Theorem 13.11 
Corollary 3.4 Consider two algorithmic short exact sequences 

1 — > F — > Gi — > H x — ► 1 
1 — ► F — > G 2 — > H 2 — ► 1 
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both satisfying the hypotheses of Theorem \3.1\ and sharing the same first term. If the two action 
subgroups coincide, Aq 1 = Aq 2 Aut(F), then G\ has solvable conjugacy problem if and only if 
G2 has solvable conjugacy problem. □ 

Note that, in the situation of Corollary 13.41 Ad and Aq 2 can be equal, even with G\ and G2 
being far from isomorphic (for example, choose two very different sets of generators for A ^ Aut(F), 
and consider two extensions of F by a free group with free generators acting on F as the chosen 
automorphisms). A nice example of this fact is the case of doubles of groups: the double of G over 
F ^ G is the amalgamated product of two copies of G with the corresponding F's identified in the 
natural way. 

Corollary 3.5 Let l^F^G^H^l be a short exact sequence satisfying the hypotheses of 
Theorem \3.1l Then G has solvable conjugacy problem if and only if the double of G over F has 
solvable conjugacy problem. 

Proof. Let 1 — > F' — > G' — > H' — ► 1 be another copy of the same short exact sequence, and 
construct the double of G over F, say G * G' . We have the natural short exact sequence for it, 

F — F' 

1 — ► F = F' — ► G * G' — > H * H' — > 1, 

F—F f 

whose action subgroup clearly coincides with Aq ^ Aut(F). So, Corollary 13.41 gives the result. □ 

4 Enlarging the applicability of Theorem 13.11 

In this section we shall enlarge the applicability of Theorem 13. 1\ by solving the twisted conjugacy 
problem for more groups F other than free and free abelian (Subsection 14. ip . and by finding more 
groups H with conditions (ii) and (iii), other than free (Subsection I4.2p . 

4.1 More groups with solvable twisted conjugacy problem 

Let us begin by proving a partial converse to Theorem 13. 1\ in the case where H = Z. In this case, 
conditions (ii) and (iii) are obviously satisfied, and the relevant part of the statement says that, 
for <p G Aut(F), solvability of TCP^r^-F) for every r E Z and of OD( (</?)) does imply solvability of 
CP(-F XI u> Z), see Remark 13.21 A weaker version of the convers is true as well. 

Proposition 4.1 Let F be a finitely generated group and let ip E Aut(F), both given in an algorith- 
mic way (and so F xuZj. If CP(F x^Zj is solvable then TCP V (F) is solvable, and {ip) ^ Aut(F) 
is orbit decidable. 

Proof. Let us assume that F x^Z has solvable conjugacy problem. Then, {ip) Aut(F) is orbit 
decidable by Theorem 13.11 (a) (c) (which uses non of the hypothesis there) applied to the short 
exact sequence 1 — > F — >■ F x„ Z — ► Z — ► 1. 

On the other hand, let u,v G F. For x G F, the equality {x(p)~ l ux = v holds in F if and only 
if x _1 (tn)x = tv holds in F Z. So, u ^ v if and only if tu and tv are conjugated to each other 
in F Z by some element of F. By hypothesis, we know how to decide whether tu and tv are 
conjugated to each other in F x^, Z by an arbitrary element, say t k x. And, 

(t k x)-\tu)(t k x) = x-H- k tut k x = x-H(uip k )x. 
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Hence, tu and tv are conjugated in F Z if and only if, for some integer k, t(u(p k ) and tv are 
conjugated to each other in F xu Z by some element of -F. That is, if and only if, wp k i; for 
some integer k. But equation u = (uip) -1 (uip)u tells us that u ~^ 1x9? (see Lemma 1.7 in [6j). So, 
solving the conjugacy problem in F xi« Z for tit and tv, is the same as deciding whether it ~„ v in 
i 7 . Thus, i 7 has solvable </?-twisted conjugacy problem. □ 

Combining Theorem 13.11 and this weak convers, we have the following remarkable consequence. 

Corollary 4.2 Let F be a finitely generated group given in an algorithmic way. All cyclic exten- 
sions F xi ^ Z have solvable conjugacy problem if and only if both F has solvable twisted conjugacy 
problem and all cyclic subgroups of Aut{F) are orbit decidable. □ 

This corollary is useful in order to find more groups with solvable twisted conjugacy problem. 
For example, using a recent result by J. P. Preaux, we can easily prove that surface groups have 
solvable twisted conjugacy problem, and that Brinkmann's result (see [10J or Theorem 16.171 in 
section [6721 below) is also valid for surface groups. 

Theorem 4.3 Let F be the fundamental group of a closed surface. Then, TCP(F) is solvable and 
all cyclic subgroups of Aut(F) are orbit decidable. 

Proof. By Theorem 1.6 in [33], F xi^ Z has solvable conjugacy problem for every automorphism 
if € Aut(F). Now, Corollary 14.21 gives the result. □ 

The same argument also provides an alternative proof of the recent result that polycyclic groups 
have solvable twisted conjugacy problem (see |18j). We remind that a group F is called polycyclic 
when there exists a finite sequence of subgroups F = Kq > K\ D> • • • > K n -\ \> K n = 1, each 
normal in the previous one, and with every quotient Ki/Ki+x being cyclic (finite or infinite). If all 
these quotients are infinite cyclic we say that F is poly-7L. 

Theorem 4.4 Let F be a polycyclic group. Then, F has solvable twisted conjugacy problem, and 
all cyclic subgroups of Aut(F) are orbit decidable. 

Proof. Let ip be an arbitrary automorphism of F. Since F is normal in F xi^Z with cyclic quotient, 
the group F Z is again polycyclic. So, it has solvable conjugacy problem (see |36j). Again, 
Corollary 14.21 gives the result. □ 

Let us study now how the twisted conjugacy problem behaves under finite index extensions. To 
do this, we shall need the classical Todd-Coxeter algorithm, and a technical lemma about computing 
finite index characteristic subgroups. 

Theorem 4.5 (Todd-Coxeter, Chapter 8 in |22j) Let L = (X \ R) be a finite presentation 
and K ^ L a finite index subgroup given by a finite set of generators. There is an algorithm to 
compute a set of left coset representatives {1 = go,gi, ■ ■ ■ ,g q } of K in L (so, L = K U g\K U 
• • • U g q K ), plus the information on how to multiply them by generators of L on the left, say 
x i9j £ 9d(i,j)K ■ Moreover, one can algorithmically write any given g € L in the form g = g p k 
for some (unique) p = 0, . . . ,q, and some k G K expressed as a word on the generators of K. In 
particular, MP(K,L) is solvable. 

Lemma 4.6 Let F be a group given by a finite presentation (X \ R), and suppose we are given 
a set of words {w\, . . . ,w r } on X such that K = (w\, . . . ,w r ) ^ F is a finite index subgroup. 
Then, the finite index characteristic subgroup Kq = D[F-i<r']=s ^ F, where s = [F : K], has a 
computable set of generators. 
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Proof. Think F as a quotient of the free group on X, tt: F{X) —» F (write N = ker7r). Using 
the elementary fact that, for every finite index subgroup L ^ F(X), [F(X) : L] ^ [F : Ltt] with 
equality if and only if N ^ L, we have 



K = f| K'= f| (K"tt) 

K%F N^K'%F(X) 



n k " 

N^K'%F(X) 



7T. 



[F:JC']=s [F(X):K"]=s \ [F(X):K"]=s J 

Hence, we can compute generators for Kq (as words in X) by first computing s (use Todd-Coxeter 
algorithm), and then listing and intersecting (and finally projecting into F) all of the finitely 
many subgroups K" of index s in F(X) which contain N: the listing can be done by enumerating 
all saturated and folded Stallings graphs with s vertices, and then computing a basis for the 
corresponding subgroup (see [39]); intersecting is easy using the pull-back technique (again, see [39J); 
and deciding whether such a K" contains N can be done, even without knowing explicit generators 
for N, by projecting K" down to F, and using Todd-Coxeter again to verify if the image has index 
exactly s, or smaller. □ 

A result by Gorjaga and Kirkinskii (see |19j ) states the existence of a group F with an index two 
subgroup K ^ F, such that the conjugacy problem is solvable in K but unsolvable in F (two years 
later the same was independently proved by Collins and Miller in [12], together with the opposite 
situation, unsolvability for K and solvability for F). In other words, the conjugacy problem does 
not go up or down through finite index extensions. In contrast with this, the following proposition 
shows that the twisted conjugacy problem does (assuming the subgroup is characteristic). 

Proposition 4.7 Let F be a group given by a finite presentation {X \ R), and suppose we are 
given a set of words {wi, . . . , w r } on X such that K = {w\, . . . ,uu r ) ^ F is a finite index subgroup. 

(i) Suppose K is characteristic in F; if TCP(K) is solvable then TCP(F) is also solvable. 

(ii) Suppose K is normal in F; if TCP(K) is solvable then CP(F) is solvable. 

Proof. By Todd-Coxeter algorithm, we can compute a set {1 = yi,y2, ■ ■ ■ ,Us} of left coset repre- 
sentatives of K in F, i.e. F = K U yiK U • • • U y s K, and use them to write any u € F (given as 
word in X) in the form yik for some i = 1, . . . , s, and some k £ K (expressed as a word in the 
ifi's); in particular, MP (If, F) is solvable (see Theorem I4.5|) . 

Suppose now that K is characteristic in F, and TCP (if) is solvable. Fix ip S Aut(F), and let 
u, v € F be two elements in F. With the previous procedure, we can write them as u = y%z and 
v = yjz', for some i,j = 1, . . . , s and z,z' £ K. Deciding whether or not u and v are ip- twisted 
conjugated in F amounts to decide whether there exist I = 1, . . . , s and k 6 K such that 

((ky^ipXvizXyik' 1 ) = yjz' 

or, equivalently, 

(yw^yizyi = {kpY l yjz'k = 2/j[(yJ" 1 (fe^)y 3 -)~VA;]. 

Now observe that, if K is characteristic in F, then the bracketed element [(y~ 1 (k<p)yj)~ 1 z'k] 
belongs to K. For every I = l,...,s, compute (yip)~~ x yizyi and check if it belongs to the 
coset corresponding to yj. If non of them do, then yiz and yjz' are not 97-twisted conjugated. 
Otherwise, we have computed the non-empty list of indices I and elements ki € K such that 
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(ynp)~ l yizyi = yjki. For each one of them, it remains to decide whether there exists k G K such 
that ki = (yj l (k(p)yj)~ 1 z'k = (kfjy :j )~ 1 z'k. This is precisely deciding whether k\ and z' are 
(t/?7 % .)-twisted conjugated in K (which makes sense because Vlyj restricts to an automorphism of 
K). This is doable by hypothesis, so we have proven (i). 

The previous argument particularized to the special case if = Id works assuming only that K 
is normal in F. This shows (ii). □ 

Theorem 4.8 Let F be a group given by a finite presentation (X \ R). If either F is (finitely 
generated) 

(i) virtually abelian, or 

(ii) virtually free, or 

(in) virtually surface group, or 
(iv) virtually poly cyclic, 
then TCP(F) is solvable. 

Proof. We first need to compute generators for a finite index subgroup K of F being abelian, or free, 
or surface, or polycyclic (and so, finitely generated, like F). We shall identify such a subgroup by 
finding a special type of presentation for it. In the first three cases, a finite presentation will be called 
canonical if the set of relations, respectively, contains all the commutators of pairs of generators, or 
is empty, or consists precisely on a single element being a surface relator. For the polycyclic case, 
let L = (yi, . . . ,y n \ S) be a given finite presentation and, for every automorphism ip € Aut(L), 
consider the cyclic extension L x v Z, and its redundant presentation C£ v {(y\, . . . , y n \ S)) given by 
L x^Z = (yi, . . . , y n ,t \ S, t~ 1 yit = Ui, tyit~ x = V{ (i = 1, . . . , n)}, where U{ and Vi are words on the 
yj's describing, respectively, the images and preimages of yi under <p. The point of considering such 
redundant presentations is that, once a particular presentation of this type is given, one can easily 
verify whether yi i— > Ui defines an automorphism of G (with inverse given by yi i— > Vi). Now every 
non-trivial, poly-Z group admits a presentation of the form C£ tpn (- ■ ■ (C£ tpi ((x \ — ))) • • • ), which 
we shall call canonical (add to this definition that the canonical presentation of the trivial group 
is the empty one). Observe that, given a presentation, it can be verified whether it is canonical in 
any of the above four cases. 

To prove the theorem, let us first enumerate the list of all subgroups of finite index in F, say 
K\, K2, .... This can be done by following the strategy in the proof of Lemma 14.61 enumerate 
all subgroups of the free group F(X) of a given (and increasing) index, and project them into F. 
For every i ^ 1, while computing ifj+i, start a new parallel computation following Reidemeister- 
Schreier process (see Section 3.2 in |27| ) in order to obtain a finite presentation for Ki, say {Yi \ Si). 
Then, start applying to (Yi \ Si) the list of all possible sequences of Tietze transformations of any 
given (and increasing) length. When one of the running processes finds a canonical presentation (of 
an abelian, or free, or surface, or poly-Z group) then stop all of them and output this presentation. 
Although many of these individual processes will never end, one of them will eventually finish 
because we are potentially exploring all finite presentations of all finite index subgroups of F and, 
by hypothesis, at least one of them admits a canonical presentation (we use here the fact that every 
polycyclic group is virtually poly-Z, see Proposition 2 in Chapter 1 of [37]). The final output of 
all these parallel processes is a canonical presentation for a finite index subgroup K of F being 
abelian, or free, or surface, or poly-Z. 
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Now, apply Lemma 14.61 to compute generators of a finite index characteristic subgroup Kq ^ F 
inside K. Note that Kq (for which we can obtain an explicit presentation by using Reidemeister- 
Schreier method) will again be either abelian, or free, or surface, or polycyclic. So, TCP(i^o) is 
solvable by results above. Hence, TCP(F) is also solvable by Proposition 14.71 (ii). □ 

The following results are two other interesting consequences of Propositions 14.71 and 14.11 

Corollary 4.9 There exists a finitely presented group G with CP(G) solvable, but TCP(G) un- 
solvable. 

Proof. Consider a finitely presented group F with an index two subgroup G ^ F, such that 
CP(G) is solvable but CP(-F) is unsolvable (see, for example, Gorjaga-Kirkinskii [19] or Collins- 
Miller [12] ). Since index two subgroups are normal, Proposition 14.71 (ii) implies that TCP(G) must 
be unsolvable. □ 

Corollary 4.10 There exists a finitely presented group G and an automorphism ip £ Aut(G) such 
that CP(G) is solvable and CP(G y\^TL) is unsolvable. Conversely, there also exists a finitely 
presented group G and an automorphism p G Aut{G) such that CP(G x^Zj is solvable and CP(G) 
is unsolvable. 

Proof. For the first assertion, consider a finitely presented group G like in the previous theorem, 
with CP(G) solvable and TCP(G) unsolvable. There must exist ip G Aut(G) with TCP^(G) 
unsolvable. Then, by Proposition 14.11 CP(G x^ Z) is unsolvable too. 

For the second assertion, start with Collins-Miller example of an index two extension G ^ F with 
unsolvable CP(G) but solvable CP(-F). In this construction, it is easy to see that F = G x^ C2 
for an order two automorphism ip G Aut(G). And solvability of CP(G x^, C<i) directly implies 
solvability of CP(G x^ Z) because the square of the stable letter belongs to the center of this last 
group. □ 

4.2 Centralizers in hyperbolic groups 

Hypotheses (ii) and (iii) in Theorem 13.11 are also satisfied by a bigger family of groups beyond 
free, including finitely generated torsion-free hyperbolic groups. All these groups H provide new 
potential applications of Theorem 13.11 

Proposition 4.11 Let H be a finitely generated hyperbolic group given by a finite presentation, 
and let h be an element of H . 

(i) There is an algorithm to determine whether or not the centralizer Cn(h) is finite and, if it is 
so, to list all its elements. 

(ii) If h has infinite order in H, then (h) has finite index in G//(/i) and there is an algorithm 
which computes a set of coset representatives for (h) in Cnih). 

Proof. From the given presentation, it is possible to compute a hyperbolicity constant 5 for H 
(see [32j or [16 J). Now, in 1.11 of Chapter HIT of [9], the authors provide an algorithm to solve 
the conjugacy problem in H (the reader can easily check that the q.m.c. property used there, holds 
with constant K = 45 + 1 in our case). In fact, the same construction (with u = v = h) also enables 
us to compute a finite set of generators for Cu{h) (as labels of the closed paths at vertex h in Q). 
So, we have computed a generating set {g\, . . . ,g r } for Cn(h). 
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On the other hand, by [5], each finite subgroup of H is conjugate to a subgroup which is 
contained in the ball of radius 45 + 1 around 1, B{48 + 1). In particular, if Cu(h) is finite then 
CH{h) x C B{45 + 1) for some x G H. We can apply a solution of the conjugacy search problem 
for H (which is solvable) to h and each one of the members of B{45 + 1), ending up with a list of 
elements {xi, . . . , x s } such that {h Xl , . . . , h Xs } are all the conjugates of h belonging to B(45 + 1). 
Now, Cff(h) is finite if and only if Cn(h) x C B{45 + 1) for some x £ H, and this happens if 
and only if Cjj(h) Xi = (gf*, . . . ,g Xl ) C B{45 + 1) for some i = 1, . . . , s (because if h y = h then 
Ci{(h) v = Cu(h)). Furthermore, this last condition, for a fixed i, is decidable in the following 
way: recursively, for I = 1,2,..., check whether the set B[ of all products of I or less (<7?*) 's is 
contained in B (4(5+1); the process will eventually find a value of I for which either B\ 5(45+1) or 
Bi+i = Bi<Z B(45 + 1). In the first case C H (h) x * = (gf\. ..,g x >)<£ B(45 + 1) (and if this happens 
for every i then Cu{h) is infinite); and in the second case Cu{h) Xl = (g^, ■ ■ ■ ,5^) != B{45 + 1), 
which means that C#(/i) is finite and, conjugating back by x" 1 , we obtain the full list of its 
elements. This concludes the proof of (i). 

By Corollary 3.10 in Chapter HIT of [9], the group (h) has finite index in Cn(h). As explained 
in the proof of that corollary, different positive powers of h are not conjugate to each other. 
Therefore, there exists a natural number k ^ |5(4<5)| + 1, such that h k is not conjugate into the 
ball B(45). In the proof it is claimed that each element of Cn(h) lies at distance at most 2\h k \ + 45 
of (h k ) and hence of (h). This means that there exists a set of coset representatives for ih) in 
Cn{h) inside the ball of radius K = 2(\B{45)\ + l)|/i| + 45. List the elements of such ball, delete 
those not commuting with h (use WP(iJ) here) and, among the final list of candidates z±, . . . , z r , 
it remains to decide which pairs Zi,zj satisfy (h)zi = (h)zj. This is the same as z = ZjZ^ 1 € (h) 
which can be algorithmically checked in the following way. For every element w of any group, the 
translation number of w is r(u>) = linin^oo — j-p, where | • | denotes the word length with respect 
to a given presentation (see Definition 3.13 in Chapter HIT of [9]). Obviously, t(w) ^ and, 
for hyperbolic groups, there exists a computable e > such that t(w) > e for every w of infinite 
order (see Proposition 3.1 in [13J, or Theorem 3.17 in Chapter HIT of [9]). Back to our situation, 
if z = h s for some integer s, then t(z) = \s\r(h) and so, \s\ = ^ Thus, the exponent s has 
finitely many possibilities and so, we can algorithmically decide whether z € (h). □ 

Theorem 4.12 Consider a short exact sequence given by finite presentations, 

1 — ► F — >G — > H — >1, 

where F is (finitely generated) virtually free, or virtually abelian, or virtually surface, or virtually 
poly cyclic group, and H is a (finitely generated) hyperbolic group where every non-trivial element 
of finite order has finite centralizer. Then, G has solvable conjugacy problem if and only if the 
corresponding action subgroup Aq ^ Aut(F) is orbit decidable. 

Proof. We just need to check that hypotheses of Theorem 13.11 are satisfied. Certainly, (i) was 
already considered above, and (ii) is well known (see 1.11 in Chapter HIT of [9]). For (iii), apply 
the algorithm given in Proposition 14.111 (i) to h ^ 1. If it answers that Cn{h) is finite, then 
the answer comes with the full list of its elements from which, using a solution to WP(-ff), we 
can extract the required list of coset representatives for Cnih) modulo {h). Otherwise, C}{{h) is 
infinite; in this case, our hypothesis ensures that h has infinite order in H, and hence we can apply 
the algorithm given in Proposition 14.111 (ii) to find the required list anyway. □ 
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5 The conjugacy problem for Aut{F2) 



The conjugacy problem for Aut(F n ) is a deep open question about free groups (a possible plan 
to solve it has been indicated by M. Lustig in the preprint (2B], and some partial results already 
published by the same author in this direction). Among other partial results, it is known to 
be solvable for rank n = 2 (see [3] or [1]). As an illustration of the potential applicability of 
Theorem 13. 1\ we dedicate this section to deduce from it another solution to the conjugacy problem 
for Aut(F 2 ). 

Consider the standard short exact sequence involving Aut(F 2 ), 

1 -» Inn(F 2 ) -> Aut(F 2 ) -» GL 2 (Z) -> 1. 

Although Inn(F 2 ) is isomorphic to F 2 , which has solvable twisted conjugacy problem (see [6]), 
Theorem 13.11 cannot be directly applied to this sequence because some centralizers in GL 2 (Z) are 
too large. Specifically, this group has a centre consisting of plus and minus the identity matrix. 
However, if we quotient out by this order two subgroup, we obtain PGL 2 (Z) which will turn out 
to satisfy our required hypotheses. Let us amend the short exact sequence above as follows. 

Choose a basis {a, 6} for F 2 and let a be the (order two) automorphism of F 2 sending a to a -1 
and b to i> -1 . Note that wo = (w -1 )^ = (u>^) -1 , where (-) R denotes the palindromic reverse of 
a word on {a, b}. Also, for every cj) 6 Aut(F 2 ), (j) R = a~ 1 (f>a acts by sending a to (a(f>) R and b to 
(b(j)) R . Consider the subgroup (Inn(F 2 ),a) ^ Aut{F 2 ). Since o~~ x <\)~ x o(\) abelianizes to the identity, 
(f)~ l o-<j) = °~lx for some x G F 2 . This (together with the elementary fact 4>~ 1 ^ w 4' = lw<f>) shows that 
(Inn(F 2 ),a) is normal in Aut(F 2 ). Moreover, it is isomorphic to F 2 x C 2 , a split extension of F 2 
by a cyclic group of order 2. This gives us another short exact sequence as follows 

1 -> F 2 x C 2 -> Aui(F 2 ) -► PGL 2 (Z) -> 1. (1) 

From the computational point of view we can think of (pQ) as given by presentations of the involved 
groups and the corresponding morphisms among them. But it is simpler to think of it as what 
is literally written: formal expressions of the type cr r w, where r = 0, 1 and w S F 2 (in one-to- 
one correspondence with cr r j w £ (Inn(F 2 ),a) < Aut(F 2 )); automorphisms of F 2 = (a, b \ ); and 
two-by-two integral matrices modulo ±Id; all with the obvious morphisms among them. 

Let us see that ([I]) satisfies hypotheses (i)-(iii) of Theorem 13.11 It is straightforward to show 
that F 2 is characteristic in F 2 x C 2 . Thus, by Proposition 14.71 (i) and solvability of TCP(i ? 2) 
(see [6]), we deduce that TCP(^2 x C 2 ) is solvable. So, our short exact sequence (pQ) satisfies (i). 
On the other hand, conditions (ii) and (iii) follow both from standard computations with two by 
two matrices, and also from considering the presentation of PGL 2 (Z) as amalgamated product 
PGL 2 (Z) = D 2 * D 3 (see, for example, page 24 in [14]). 

Hence, ([I]) is a short exact sequence satisfying the hypotheses of Theorem 13.11 This way, the 
solvability of the conjugacy problem for Aut(F 2 ) is equivalent to the orbit decidability of the action 
subgroup A = {i^s \ eft € Aut(F 2 )} sj Aut(F 2 x C 2 ). Thus, the following proposition provides a 
solution to CP (Aut(F 2 )): 

Proposition 5.1 With the above notation, the action of Aut{F 2 ) by right conjugation on its normal 
subgroup (Inn(F 2 ),o~) = F 2 x C 2 is orbit decidable. 

Proof. The action of Aut{F 2 ) on Inn(F 2 ) is determined by the natural action on F 2 , namely 
4>~ l lw4> = lw4>, f° r every <p € Aut(F 2 ) and w G F 2 . So, by the classical Whitehead algorithm 
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(see Proposition 1.4.19 in [25]), it is possible to decide whether two elements in Inn(F2) lie in 
the same Aut{F2)-oih\i (i.e. the action of Aut(F2) on Inn(F2) is orbit decidable). But, Inn^F-z) 
has index 2 in (Inn(F2),cr) so, it only remains to decide whether two given elements of the form 
vim &lv G (Iren(i<2), cr), with u, v G F2, lie in the same ^lu^i^-orbit or not. 

Suppose then that aj u and aj v are given to us, and let us algorithmically decide whether there 
exists 4> G Aut{F2) such that 4>~ l a^ u 4> = o"7„. 

Note that, if this is the case, then (j) -1 (a^/ u ) 2 <p = {&lv) 2 , while ((tj u ) 2 and (a^ v ) 2 belong to 
Inn(F2). So, apply Whitehead algorithm to (o"7«) 2 and (<77„) ; if they are not in the same Aut{F2)- 
orbit, the same is true for aj u and crj v , and we are done. Otherwise, we come up with a particular 
a G Aut(F2) such that a~ 1 (aj u ) 2 a = (a^/ v ) 2 ; now, replacing a^ u by a -1 <77 u a, we can assume that 
(°"7m) 2 = {^Iv) 2 ■ This means 

7(u<t)u = <?lu<?lu = {<?luf = (vjv) 2 = O^ivOlv = 7(w)tn 

i.e. (ua)u = (vcr)v. We can compute this element of i*2 and distinguish two cases depending 
whether it is trivial or not: 

Case 1: [ua)u = (va)v = 1. Note that, in this case, u and v are palindromes (i.e. u R = u and 
v R = v). We shall show that aj u and a^/ v are always in the same Aut(F2)-ovbit. In fact, notice 
that if x G {a, 6} ±1 is any letter then, 

Ix 1 {°'lu)lx = J x -i(TJu7x = CFlxux- 

Hence, we can consecutively use conjugations of this type to shorten the length of u (and v) down 
to or 1; that is, aj u (and (J^ v ) is in the same Aut{Fi)-oxbit as at least one of a, aj a , cy a -i , 0-7^ or 
0"7 6 -i. But, r )a 1(J la- 1 la = &la and 7 fc ~ 1 (77 b -i7b = o"76. Also, defining p G Aut{F2) as a 1— > 6, 6 1— ► a, 
we have p~ l a^ a p = ajf Finally, defining \ £ Au£(.F2) as a 1— > a&, 6 1— > b, we have X~ 1(7 X = cr 7fe- 
Thus, o"7 u and cry v are always in the same Aut(F2)-orbit in this case. 

Case 2: (ua)u = (vcr)v 7^ 1. Let z be its root, i.e. (ua)u = (va)v = z s for some s ^ 1, with 
z not being a proper power. If some <fi G Aut{F2) satisfies </> _1 <77 u </> = 177,; then such <j) must also 
satisfy 

= 4>~ X lz^ = _1 (cr7 u ) 2 = (aj v ) 2 = 7 Z « 

and so, z<p = z. In other words, a"7 n can only be conjugated into a^ v by automorphisms stabilizing 
z. Using McCool's algorithm (see Proposition 1.5.7 in [25]), we can compute a set of generators 
for this subgroup of Aut{F2), say Stab(z) = (4>i, . . . , <pk)- 

It remains to analyze how those 0j's act on aj u . We can compute <\\ X (5^iu4>i and write this 
element of F2 x C2 in normal form, say crj Wi . We claim that wi = (za) ni u for some G Z. In fact, 
squaring <J^ u (t>i = CTui* (and using that <pi stabilize z) we obtain 

l(uu)u = {Oluf = lz° = 4>J ln lz^i = ^ l {criu) 2 4>i = (O-Jwtf = 7(w i0 >i- 

This implies z s = (wx)u = (wia)wi and, applying a to both sides, z s a = u(ua) = Wi(wicr). Now 
observe that 

(wiu~ l )^ z s a = (z s a)' 1 (w i u~ 1 )(z s a) = (w^ 1 o)wJ 1 {w i u' 1 )u{uu) = (w^a^ua) = WiU' 1 , 

which means that wiu^ 1 commutes with z s a = (za) s . Hence, WiU~ l = (za) ni and wi = (za) ni u 
for some computable n; G Z. We have shown that, for i = 1, . . . , k, 
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Few more computations show that conjugating by (ft i makes the corresponding negative effect on 
the exponent: from <ft^ 1 aj u (fti = °~l(za) n iu = lz n iO~y u we deduce 

And conjugating by another cftj makes an additive effect on the exponent: 

= <t>j 1 1z n i&Ju4>j 

= iz n i4>~j 1 °iu4>j 

= a 7(za) n * +n Ju 

So, conjugating by (ft € Stab(z) ^ Aut(F2), we can only move from aj u to elements of the form 
a l{za) Xn oui where no = gcd(ni, . . . , n^), and A G Z. Thus, 177^ and aj v belong to the same Aut{F2)- 
orbit if and only if for some A € Z, which happens if and only if vu 1 is a power of 

(za) n ° . This is decidable in a free group. □ 

Corollary 5.2 Aut(F2) has solvable conjugacy problem. □ 

6 Positive results 

In this section, positive results for the free and free abelian cases are analyzed. Along the two paral- 
lel subsections, we will give several examples of orbit decidable subgroups in Aut(7* n ) = GL n (7j) and 
Aut(F n ), together with the corresponding [f.g. free abelian]-by-[f.g. free] and [f.g. free]-by-[f.g. free] 
groups with solvable conjugacy problem (see Theorem 13. 3p . The reader can easily extend all these 
results to [f.g. free abelian]-by- [f.g. t.f. hyperbolic] and [f.g. free] -by- [f.g. t.f. hyperbolic] groups, by 
direct application of Theorem 14.121 (here, t.f. stands for torsion free). 

As a technical preliminaries, we first need to solve the coset intersection problem for free and 
virtually free groups, and to see that orbit decidability goes up to finite index. 

Proposition 6.1 Let K be the free group with basis X. Then, the coset intersection problem 
CIP(K) is solvable. 

Proof. Without loss of generality, we can assume that X is finite, since CIP(K) only involves two 
finitely generated subgroups A,B^K and two words x,y G K. Now, using Stallings' method 
(see [39]), we can construct the corresponding core-graph Tx(A) (resp. Tx(B)) and attach to it, 
and fold if necessary, a path labelled x (resp. y) from a new vertex v x (resp. v y ) to the base-point 
1 in Tx{A) (resp. 1 in Tx{B)). Now, after computing the pull-back of these two finite graphs, we 
can easily solve the coset intersection problem: elements from xA n yB are precisely labels of paths 
from (v x ,v y ) to (1, 1) in the pull-back. So, xA n yB ^ if and only if (v x , v y ) and (1, 1) belong to 
the same connected component of this pull-back. □ 

Proposition 6.2 Let L be a group given by a finite presentation (X \ R), and let K be a finite 
index subgroup of L generated by a given finite set of words in X. If CIP(K) is solvable then 
CIP(L) is also solvable. 
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Proof. By Todd-Coxeter algorithm (see Theorem 14. 5p . we can compute a set of left coset represen- 
tatives {1 = go, gi, . . . , g q } of if in L (so, L = if U g\K U • • • U g q K), plus the information on how 
to multiply them by generators of L on the left, say Xigj G g^n^K. This information can be used 
to algorithmically write any given g G L in the form g = g p k for some (unique) p = 0, . . . , q, and 
k G if. In particular, MP(if, L) is solvable. 

Suppose now several elements x,y, ax, . . . , a r , bx, . . . , b s G L are given; we shall show how to 
decide whether xA n yB is empty or not, where A = (ai, . . . , a r ) ^ L and B = (6i, . . . , b s ) ^ L. 

Let us compute a (finite) set of left coset representatives W of A modulo ^4 n if in the following 
way. Enumerate all formal reduced words in the alphabet {ax, ■ ■ ■ , ar}^ 1 , say {w\, ■ ■ •}, starting 
with the empty word, wx = 1, and in such a way that the length never decreases. Now, starting with 
the empty set {7 = 0, recursively enlarge it by adding Wi whenever Wi(A n if) 7^ w^(A n if) for all 

G f7 (use here MP (If, L)). Since if has finite index in L, Af)K has finite index in A and the set 
U can grow only finitely many times. Stop the process at the moment when, for some I, no word of 
length I can be added to U. At this moment we have exhausted the search inside the Schreier graph 
of A modulo AnK, and the existing list U = {1 = ux, ■ ■ ■ , u m } is a set of left coset representatives of 
AnK in A, say A = [J™^ Ui(AnK). Now, for every a = 1, . . . , r and i = 1, . . . ,m, compute d(a, i) 
such that a a Ui(A n K) = u d ( a ^(A n K) (again using MP(if, L)). By the Reidemeister-Schreier 
method (see Theorem 2.7 in [27]), the set {uZ a ^a a Ui \ a = 1, . . . , r, i = 1, . . . , m} generates AnK. 
Analogously, we can compute a set V = {vx, ■ ■ ■ , v n } of left coset representatives for B n K in B, 
say B = |_l™ = i v j(B n if), together with a set of generators for B n if . Clearly, 

xAHyB= (\Jxui(AnKU n ( LJy^-(snif) ] =U(icu i (Anif)ny^(Bnif)). 
\i=i / \j=i / i,j 

Hence, n y-B 7^ is equivalent to xui(yl n if) D yvj(B n if) 7^ for some i = 1, . . . , m and some 
j = 1, ... ,n. For each consider the element 2y = vj 1 y~ 1 xiii and rewrite it in the form <7 p fc 

for some (unique) p = 0, . . . , q and fc G if . If p 7^ (i.e. if Zjj does not belong to if) then the 
intersection xui{A n if) fl yvj(B n if) is empty. Otherwise, Zjj G if and we are reduced to verify 
whether Zij(A n if ) n (B D if ) 7^ 0. This can be done using CTP(if). □ 

Let us apply this result to GL?ffi)- It is well-known that GL^i^) admits the following presen- 
tation 

GL 2 {Z) =s L> 4 * As = <*4, ^4 I *i xt, to) 2 ) * {t e , x 6 \ tj, x%, {t 6 x 6 f), (2) 

Da / *4 = *6 \ 

\ x| = x| / 

where i 4 = t6 = (io)> x 4 = (1 V) an< ^ x& = (1 V) ( see Example 1.5.2 in [14J). Since one 
can algorithmically go from matrices to words in the presentation, and viceversa, both models are 
algorithmically equivalent (and we shall use one or the other whenever convenient). 

Corollary 6.3 The problem CIP(GL 2 {Z)) is solvable. 

Proof. There is a natural epimorphism <p: GL2^L) -» Dx2 = {tx2, x i2 \ tx2> x xh (^12^12) 2 ) given by 
ti = te 1— > tx2 j Xi 1— ► x\ 2 , %6 l— ► ^12- With a few straightforward calculations, one can show that 
if = ker y> is free of rank 2, with basis P = [xq, X4] = ( \ \) and Q = [ccg, X4] = ( \ \ ). Since if has 
index I-D12I = 24 in L = GL2CZ), Propositions 16. l l and 16.21 conclude the proof. □ 
Finally we see that, in general, orbit decidability goes up to finite index. 
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Proposition 6.4 Let F be a group given in an algorithmic way, and let A ^ B ^ Aut(F) be two 
subgroups given by finite sets of generators, such that A has finite index in B and MP(A,B) is 
solvable. If A ^ Aut(F) is orbit decidable, then B ^ Aut{F) is orbit decidable. 

Proof. With a coset enumeration argument similar to that in the proof of Proposition 16.21 we 
can compute a (finite) list, say . . . , /3 m }, of left coset representatives of A in B, i.e. B = 
(3\A U • • • U (3 m A (we formally need MP(A, B) here, because B may have infinite index in Aut(F)). 
Now, for any given u,v G F, the existence of (5 G B such that u(3 is conjugate to v is equivalent to 
the existence of i = 1, . . . , m and a G A such that {u(3i)a is conjugate to v. Hence, orbit decidability 
for B ^ Aut(F) follows from orbit decidability for A ^ Aut{F). □ 

6.1 The free abelian case 

Let us concentrate on those short exact sequences in Theorem 13.11 with F being free abelian, say 
F = Z n , and look for orbit decidable subgroups of Aui(Z n ) = GL n (Z). 

To begin, it is an elementary fact in linear algebra that two vectors u, v G Z n lie in the same 
GL n (Z)-orbit if and only if the highest common divisor of their entries coincide and, in this case, 
with the help of Euclid's algorithm, one can find an invertible matrix A such that uA = v. In other 
words, 

Proposition 6.5 The full automorphism group GL n (Z) of a finitely generated free abelian group 
7L n , is orbit decidable. □ 

Corollary 6.6 Let A 1} . . . , A m G GL n (Z). If (A l ,..., A m ) = GL n (Z), then the Z n -by-F m group 
G = Z n Xj4i,...,A m Fm has solvable conjugacy problem. □ 

It is also a straightforward exercise in linear algebra to see that cyclic subgroups of GL n (Z) 
are orbit decidable. That is, given A G GL n (Z) and u,v G Z" one can algorithmically decide 
whether uA k = v for some integer k: in fact, think A as a complex matrix, work out its Jordan 
form (approximating eigenvalues with enough accuracy) and then solve explicit equations (with the 
appropriate accuracy). This provides a solution to the conjugacy problem for cyclic extensions of 
Z n . 

Proposition 6.7 Cyclic subgroups of GL n (Z) are orbit decidable. □ 
Corollary 6.8 Z n -&?/-Z groups have solvable conjugacy problem. □ 

However, this was already known via an old result due to V.N. Remeslennikov, because Z n - 
by-Z groups are clearly polycyclic. In [36] it was proven that polycyclic groups G are conjugacy 
separable. As a consequence, such a group, when given by an arbitrary finite presentation, has 
solvable conjugacy problem (use a brute force algorithm for solving CP + (G), and another for 
CP _ (G) enumerating all maps into finite symmetric groups (i.e. onto finite groups) and checking 
whether the images of the given elements are conjugated down there). But, furthermore, this result 
can now be used to prove a more general fact about orbit decidability in GL n (Z). 

J. Tits [ID] proved the deep and remarkable fact that every finitely generated subgroup of 
GL n (Z) is either virtually solvable or it contains a non-abelian free subgroup. The following propo- 
sition says that the first kind of subgroups are always orbit decidable, so forcing orbit undecidable 
subgroups of GL n (7j) to contain non-abelian free subgroups. This somehow means that orbit un- 
decidability in GL n (Z) is intrinsically linked to free-like structures. 
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Proposition 6.9 Any virtually solvable subgroup of GL n (Z) is orbit decidable. 

Proof. Let 5 be a virtually solvable subgroup of GL n (X), given by a finite generating set of 
matrices Ai, . . . ,A m . By a Theorem of A.I. Mal'cev (see [28j, or Chapter 2 in [37]), every solvable 
subgroup of GL n (Z) is polycyclic; so, B = (Ai, . . . , A m ) has a finite index subgroup C ^ B which 
is polycyclic and, in particular, finitely presented. 

Recurrently perform the following two lists: on one hand keep enumerating all finite presen- 
tations of all polycyclic groups (use a similar strategy as that in the proof of Theorem 14.81 above, 
enumerating first all canonical presentations of such groups, and diagonally applying all possible 
Tietze transformations to all of them). On the other hand, enumerate a set of pairs (C,M), where 
C is a finite set of generators for a finite index subgroup C of B, and A4 = {Mi, . . . , M r } is a finite 
set of matrices such that B = Mi • C U • • • U M r • C, and in such a way that C eventually visits 
all finite index subgroups of B; we can do this in a similar way as in the proof of Lemma 14. 6t 
enumerate all saturated and folded Stallings graphs with increasingly many vertices over the al- 
phabet {Ai, . . . , A m }, and map the corresponding finite index subgroup and finite set of coset 
representatives (one for each vertex) down to B, where possible repetitions may happen (see [39J). 

These two lists are infinite so the started processes will never end; but, while running, keep 
choosing an element in each list in all possible ways, say C = (t%, . . . , t p \ R±, . . . , R q ) and (C, M), 
and check whether there is an onto map from {t±, . . . ,t p } to C that extend to an (epi)morphism 
C' — ► C (this just involves few matrix calculations in GL n (X)). Stop all the computations when 
finding such a map (which we are sure it exists because some finite index subgroup C ^ B is 
polycyclic, and so isomorphic to one of the presentations in the first list). When this procedure 
terminates, we have got a finite presentation (t\, . . . , t p \ Ri, . . . , R q ) of a polycyclic group C" and 
a map C — > C onto a finite index subgroup C ^ B = (A\, . . . , A m ) ^ GL n (Z), for which we also 
know a finite set of coset representatives M, with possible repetitions. 

Now, write down the natural presentation of the group G = 7L n ~xq C 1 . Since it is clearly 
polycyclic, Remeslennikov's result [36] tells us that G has solvable conjugacy problem (for instance, 
from the computed presentation). Thus, by Theorem 3.1 (a) => (c), the corresponding group of 
actions, C ^ GL n (Z), is orbit decidable (note that hypothesis (hi) of Theorem 3.1 may not be 
satisfied in this case, but it is not used in the proof of implication (a) (c)). Finally, B GL„(Z) 
is orbit decidable as well: given two vectors u,v € Z n , deciding whether v = uP for some P £ B is 
the same as deciding whether v = uMiQ for some i = 1, . . . , r and Q € C, which reduces to finitely 
many claims to the orbit decidability of C ^ GL n (X). □ 

Corollary 6.10 Let A\, . . . , A m S GL n (Z). If B = (A\, . . . , A m ) ^ GL n (Z) is virtually solvable, 
then the Z n -by -F m group G — ^ A\,...,A m has solvable conjugacy problem. □ 

Let us consider now finite index subgroups of GL n (Z). 

Proposition 6.11 Any finite index subgroup of GL n (Z) (given by generators) is orbit decidable. 

Proof. Let B ^ GL n (Z) be a finite index subgroup generated by some given matrices. Take your 
favorite presentation for GL n (Z) (see, for example, Section 3.5 of [27]) and write them in terms 
of it. With a similar argument as in the proof of Lemma 14.61 we can compute generators for the 
subgroup 

A= p| (p- 1 -B-P) = Br\B Pl n---nB Pm <GL n (z), 

PeGL„(Z) 
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where Id = Pq, P±, . . . , P m is a set of right coset representatives for B in GL n (Z) (computable 
by Todd-Coxeter algorithm, see Theorem I4.5|) . By Proposition 16.41 we are reduced to see that 
A < GL n (Z) is orbit decidable. 

Given u, v £ Z n , we have to decide whether some matrix of A sends u to v. Clearly, we 
can assume u,v ^ and check whether there exists M € GL n (Z) such that uM = v (see Proposi- 
tion [63]). Once we have such M, the set of all those matrices carrying u to v is precisely M -Stab(v). 
And it is straightforward to compute a finite generating set for the stabilizer of v (it is conjugate 
to Stab(l,0, ... ,0) = {(^ 2 '* *)})• It remains to algorithmically decide whether the intersection 
A n (M • Stab(v)) is or is not empty; or, equivalently, whether M € A ■ Stab(v) holds or not. This 
is decidable because A ■ Stab(v) is a finite index subgroup of GL n (Z) (here is where normality of A 
is needed) with a computable set of generators; hence, MP (.A • Stab(v), GL n (Z)) is solvable, again 
by Todd-Coxeter algorithm. □ 

Corollary 6.12 Let A\, . . . , A m € GL n (Z). If (A\, . . . , A m ) has finite index in GL n {7j) then the 
Z n -by-F m group Z n XAi,...,A m F m has solvable conjugacy problem. □ 

Finally, let us concentrate on rank two. 

Proposition 6.13 Every finitely generated subgroup of GL2(Z) is orbit decidable. 

Proof. Let A±, . . . , A r £ GL n (Z) be some given matrices and consider the subgroup they generate, 
(Ai,...,A r ) ^ GL n (7j). For n = 2, given u, v £ Z n , let us decide whether there exists A € 
{A\, . . . , A r ) such that uA = v. We can clearly assume u,u/0. 

By Proposition 16.51 we can decide whether there exists M £ GL n (Z) such that uM = v and, 
in the affirmative case, find such an M . And it is straightforward to find a set of generators for 
the stabilizer of v, Stab(v) = {B G GL„(Z) | vB = v}, say {B\, . . . , B s } (in the case n = 2, 
every such stabilizer is conjugate to Stab(l,0) = ((} ?) , (o -i)))- Then, the matrices sending u 
to v are precisely those contained in the coset M(B\, . . . ,B S ). So, it remains to decide whether 
(Ai,...,A r )n M{Bi, ... ,B S ) is empty or not. 

In the case n = 2, this can be done algorithmically (see Corollary 16. 3ft , □ 

Note that the proof of Proposition 16.131 works for every dimension n except at the end, when 
Corollary 16.31 is used. We shall refer to this fact later. 

Corollary 6.14 All Z 2 -by- [f.g. free] groups have solvable conjugacy problem.^ 
6.2 The free case 

Following the same route as in the previous subsection, let us concentrate now on those short exact 
sequences in Theorem 13.11 with F being free, say F = F n , and look for orbit decidable subgroups 
of Aut(F n ). 

To begin, classical Whitehead algorithm (see Proposition 1.4.19 in [25]) decides, given u, v G F n , 
whether there exists an automorphism of F n sending u to v up to conjugacy. In other words, 

Theorem 6.15 (Whitehead, |41j) The full automorphism group Aut{F n ) of a finitely generated 
free group F n , is orbit decidable. □ 

Corollary 6.16 Let F n be a finitely generated free group. If ipi, . . . ,<p m generate Aut(F n ), then 
the F n -by-F m group G = F n „. ^ F m has solvable conjugacy problem. □ 
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Like in the abelian case, cyclic subgroups of Aut{F n ) are orbit decidable by a result of P. 
Brinkmann. This is the analog of Proposition 16.71 for the free case, but here the proof is much more 
complicated, making strong use of the theory of train-tracks. This was already used to solve the 
conjugacy problem for free- by-cyclic groups: 

Theorem 6.17 (Brinkmann, [10J ) Cyclic subgroups of Aut(F n ) are orbit decidable. □ 

Corollary 6.18 (Bogopolski-Martino-Maslakova- Ventura, |6j) [f.g. free]-by-cyclic groups ha- 
ve solvable conjugacy problem. □ 

The analog of Proposition 16.91 and Corollary 16. 101 in the free setting is not known, and seems to 
be an interesting and much more complicated problem. See Question 5 in the last section for some 
comments about it, and a clear relation with Tits alternative for Aut(F n ). 

Let us now consider finite index subgroups of Aut(F n ). 

Proposition 6.19 Let F n be a finitely generated free group. Any finite index subgroup of Aut(F n ) 
(given by generators) is orbit decidable. 

Proof. Let B ^ Aut(F n ) be a finite index subgroup generated by some given automorphisms. 
Consider Nielsen's presentation of Aut(F n ) (see Proposition Nl in Section 3.5 of |27j ) and write 
them in terms of this presentation (i.e. as products of Nielsen automorphisms). Then, with a 
similar argument as in the proof of Lemma 14.61 we can compute generators for the subgroup 

A= p| {4>~ X Bcj)) =Br\B+ 1 n-n5*™ < Aut{F n ), 

(j>£Aut(F n ) 

where Id = (fro, (fr\, . . . , (fr m is a set of right coset representatives for B in Aut(F n ) (computable 
by Todd-Coxeter algorithm, see Theorem 14. 5p . By Proposition 16.41 we are reduced to see that 
A < Aut{F n ) is orbit decidable. 

Let u,v E F n . Using Whitehead's algorithm, we can check whether there exists an automor- 
phism a G Aut(F n ) carrying u to v. Once we have such a, the set of all those automorphisms 
carrying u to a conjugate of v is precisely a ■ Stab(v) ■ Inn(F n ). By McCool's algorithm (see 
Proposition 1.5.7 in [25j), we can compute a finite generating set for the stabilizer of v. It re- 
mains to algorithmically decide whether the intersection A n (a ■ Stab(v) ■ Inn{F n )) is or is not 
empty; or, equivalently, whether a G A ■ Stab(v) ■ Inn{F n ) holds or not. This is decidable because 
A ■ Stab(v) ■ Inn(F n ) is a finite index subgroup of Aut(F n ) (here is where normality of A is needed) 
with a computable set of generators; hence, MP(j4 • Stab(v) ■ Inn(F n ), Aut{F n )) is solvable, again 
by Todd-Coxeter algorithm. □ 

Corollary 6.20 Let F n be a finitely generated free group. If tpi, . . . , ip m generate a finite index 
subgroup of Aut{F n ), then the F n -by-F m group G = F n ^ vx ^..., Vm F m has solvable conjugacy problem. 
□ 

Now, let us concentrate on rank two. Like in the abelian case, we have 

Proposition 6.21 Let F2 be the free group of rank two. Then every finitely generated subgroup of 
Aut^F^) is orbit decidable. 
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Proof. Let A be a finitely generated subgroup of Aut(F n ) and let u,v E F n be given. For n = 2, 
we have to decide whether there exists (p 6 A such that u</? is conjugate to v. 

Mimicking the proof of Proposition 16 . 131 let us apply Whitehead's algorithm to u, v (see Propo- 
sition 1.4.19 in [25]). If there is no automorphism in Aut(F n ) sending u to v then, clearly, the 
answer to our problem is also negative. Otherwise, we have found a 6 Aut(F n ) such that ua = v. 
Now, the set of all such automorphisms of F n is a ■ Stab(v). And the set of all automorphisms of F n 
mapping u to a conjugate of v is a ■ Stab(v) ■ Inn(F n ) . By McCool's algorithm (see Proposition 1.5.7 
in [25]), we can find a finite system of generators for Stab(v) ^ Aut(F n ). Finally, all we need is to 
verify whether A n (a ■ Stab(v) ■ Inn(F n )) is or is not empty. 

In the case n = 2 this can be done algorithmically: since the kernel of the canonical projection 
— : AutiF^) -» GL2(Z) is Inn(F2) (which is a very special fact of the rank 2 case), our goal is 
equivalent to verifying whether A n (a • Stab(v)) is or is not empty in GL^i^)- This can be done 
by Corollary 16.31 □ 

Note that the proof of Proposition 16.211 works for every rank n except for the last paragraph, 
exactly like in Proposition 16.131 We shall refer to this fact later. 

Corollary 6.22 All F2-by-[f.g. free] groups have solvable conjugacy problem. □ 

Another nice examples of orbit decidable subgroups in Aut(F n ) come from geometry. Certain 
mapping class groups of surfaces with boundary and punctures turn out to embed in the automor- 
phism group of the free group of the appropriate rank. The image of these embeddings are easily 
seen to be orbit decidable in two special cases. From [15] we extract the following two examples. 

Let Sg^^n be an orientable surface of genus g, with b boundary components and n punctures. 
It is well known that its fundamental group has presentation 

£<?,b,n = (^1)2/1) • ■ -, x 9,y g , Zl, ■■■,Zb,h,...,t n \ [x 1 ,y 1 ] ■ ■ ■ [x g ,y g ]zi ■ ■ ■ z h t x •■■t n = l), 

and, except for b = n = 0, is a free group of rank 2g + b + n — 1. In the following cases, the mapping 
class group of S g ^, n can be viewed as a subgroup of Aut(F2 g +b+n-i) , see [15] for details: 

Proposition 6.23 (see [15]) Let Sg^.n be an orientable surface of genus g, with b boundary com- 
ponents, and with n punctures. 

(i) (Maclachlan) The positive pure mapping class group of 5^,0^+2 becomes (when the basepoint 
is taken to be the (n + 2 )nd puncture) the group Aut + Q lXn+1 of automorphisms o/E 9] o,n+i — 
F2g+ n which fix each conjugacy class [tj], j = 1, . . . , n+ 1 (the case g = gives the pure braid 
group onn + 1 strings modulo the center, B n+ i/Z(B n+ i)). 

(ii) (A'Campo) The positive mapping class group of S gi i )Tl becomes (when the basepoint is taken 
to be on the boundary) the group Aut^~^ of automorphisms o/S 9j i in ~ F2 g + n which fix z and 

permute the set of conjugacy classes {[ii], . . . , [t n ]} (the case g = gives the braid group on 
n strings, B n ). 

A particularly interesting case is when g = in (ii) above: Aut^ is the image of classical 

Artin's embedding of the braid group on n strings into Aut(F n ), sending generator o~i G B n (i = 
1, . . . , n - 1) to Oi : F n — > F n , ti i-> t,it i+ it~ l , t i+1 h-> U, tj h-> tj for j ^ i, i + 1. The subgroup 
Aut^^ = (a%, . . . ,c n _i) ^ Aut(F n ) is then characterized as those automorphisms 92 G Aut{F n ) 
for which (^1^2 ■ ■ ■ t n )(p = t\t2 ■ ■ ■ t n and there exist words w±, . . . , w n £ F n and a permutation a of 
the set of indices such that ti<p = w^ l t a u\Wi. 
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All these groups of automorphisms, Aut ,± n+1 ^ Aut(F2 g + n ) and Aut ~ ^ Aut(F2 g + n ), are 
easily seen to be orbit decidable because of the following observation. 

Proposition 6.24 Let F n be a finitely generated free group, and let Ui,Vi S F n (i = 0, . . . ,m) be 
two lists of elements. Then, 

(i) A = {ip | u p = vq, unp ~ vi, . . . ,u m p ~ v m } < Aut(F n ) and 

(ii) B = {(p | u ip = v , ui<p ~ . . . ,u m tp ~ v^m) for some a G Sym(m)} ^ Ant(F n ) 
are oroii decidable. 

Proof. For (i), given ii,i> E .F n , we have to decide whether there exists an automorphism ip G 
Aut(F n ) such that u^p = vq, [u{\ip = [vi], . . . , [u m ]p = [v m ] and [u]ip = [v], where brackets denote 
conjugacy classes. This is the same as deciding whether there exists ip such that [uo]p = [vo], 
[ui]<p = [vi], . . . , [u m ]ip = [v m ] and [u]ip = [v] (and, in the affirmative case, composing <p by 
7 -1, where wo is the first conjugator, uqp = vqWq). One can make this decision by applying 
Proposition 4.21 in [25]. Finally, (ii) can be solved by using up to ml many times the solution given 
for (i). □ 

It is worth mentioning that D. Larue in his PhD thesis [23] analyzed the Aut^~ ^-orbit of ii in 

So,i,n (i- e - the i? n -orbit of t\ in F n ) and he provided an algorithm to decide whether a given word 
w G F n belongs to this orbit (note that this is not exactly a special case of OT){B n ) because B n 
does not contain all inner automorphisms). Although working only for the orbit of t\, the algorithm 
provided is faster and nicer than that provided in Proposition 16.24] 

Corollary 6.25 Let i<2g+n be a finitely generated free group. Iftpi,..., <p m € Aut{F2 g + n ) generate 
the positive pure mapping class group Aut~^ Q 1±n+1 , or the positive mapping class group Aut^ n 

then the F2 g + n -by-F m group G = i^s+n ^<pi,...,tp m Fm has solvable conjugacy problem (a particular 
case of this being when (pi, . . . ,<p m generate the standard copy of the braid group B n ^ Aut(F n )). 
□ 

7 Negative results 

Let us construct now negative examples, namely orbit undecidable subgroups of GL n (Z) and 
Aut(F n ) which, of course, will correspond to [f.g. free abelian]-by-[f.g. free] and [f.g. free]-by-[f.g. free] 
groups with unsolvable conjugacy problem. 

As mentioned above, Miller constructed a [f.g. free]-by-[f.g. free] group with unsolvable conjugacy 
problem (see [30]); here, we have a first source of examples of finitely generated subgroups of the 
automorphism group of a free group, which are orbit undecidable. In the present section, we will 
generalize this construction by giving a source of orbit undecidability in Aut(F) for more groups F. 
When taking F to be free, this will reproduce Miller's example; when taking F = Z n for n ^ 4, we 
will obtain orbit undecidable subgroups in GL n (Z), which correspond to the first known examples 
of [f.g. free abelian]-by- [f.g. free] groups with unsolvable conjugacy problem. 

Let us recall Miller's construction. It begins with an arbitrary finite presentation, H = 
(s\,...,s n j R\, . . . , R m ), where the Rj's are words on the Sj's. Let F n+ \ = (q, s%, . . . , s n \) 
and F m+n = (t\,..., t m , d\, . . . , d n \ ) be the free groups of rank n + 1 and m + n, respectively, on 
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the listed generators. Consider now the m + n automorphisms of F n+ \ given by 

di : F n+ \ — ► F n+ i f3j : F n+ \ — » F n+ i 

q ^ qRi q h-> s~ l qsj , 

s k h-> Sjfc Sfc ^ s fc 

for 2 = 1, . . . , m and k = 1, . . . , n, and denote the group of automorphisms they generate by 
A(H) ^ Aut(F n+ i). Next, consider the F n+ i-by-F m+n group defined by these automorphisms, 

G(H) = F n+1 ^ai,...,a m ,/3i,...,/3„ F m+n . 

The following Theorem is Corollary 5 in Chapter III of [30]. Below, we shall provide an alternative 
proof. 

Theorem 7.1 (Miller, |30j) If H has unsolvable word problem then G{H) has unsolvable conju- 
gacy problem. 

So, applying Miller's construction to a presentation H with n generators, m relations, and with 
unsolvable word problem, one obtains a (m + n)-generated subgroup of Aut(F n+ i), namely A(H), 
which is orbit undecidable. 

In [8], V. Borisov constructed a group presented with 4 generators, 12 relations, and having 
unsolvable word problem. In order to reduce the number of generators to 2 (and so have the 
corresponding orbit undecidable subgroup living inside Aut(F3)) we can use Higman-Neumann- 
Neumann embedding theorem, saying that any countable group can be embedded in a group with 
two generators and the same number of relations (see [H]). Since solvability of the word problem 
clearly passes to subgroups, we obtain a group with n = 2 generators, m = 12 relations, and having 
unsolvable word problem. Using Miller's construction we conclude the existence of a F^-by-F^ 
group with unsolvable conjugacy problem. In other words, 

Corollary 7.2 There exists a ^-generated subgroup A ^ Aut(F^) which is orbit undecidable. □ 

Let us now find a more general source of orbit undecidability that will apply to more groups F 
other than free (and, in the free case, will coincide with Miller's example via Mihailova's result). 
Let F be a group. Recall that the stabilizer of a given subgroup K ^ F, denoted Stab(K), is 

Stab (K) = {ipe Aut(F) \ ktp = k \/k £ K} ^ Aut(F). 

For simplicity, we shall write Stab(k) to denote Stab ((k)), k E F. Furthermore, we define the 
conjugacy stabilizer of K, denoted Stab*(K), to be the set of automorphisms acting as conjugation 
on K, formally Stab*{K) = Stab(K) ■ Inn(F) < Aut{F). 

Proposition 7.3 Let F be a group. Suppose we are given two subgroups A ^ B ^ Aut(F) and an 
element v E F such that B n Stab*(v) = {Id}. If A ^ Aut(F) is orbit decidable then MP(A,B) is 
solvable. 

Proof. Given ip E B ^ Aut(F), let us decide whether tp E A or not. Take w = vij) and observe that 
{<P E B | v<j> ~ w} = B n (Stab*(v) ■ if>) =(Bfl Stab*(v)) ■ = {rp}. 
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So, there exists <f> £ A such that v<p is conjugate to w in F, if and only if ijj G A. Hence, orbit 
decidability for A < Aut(F) solves MP(vl,J3). □ 

One can interpret Proposition 17.31 by saying that if, for a group F, Aut(F) contains a pair of 
subgroups A ^ B ^ with unsolvable MP(^4, B) then A ^ Aut{F) is orbit undecidable. 

The most classical example of unsolvability of the membership problem goes back to fifty years 
ago. In [29] (see also Chapter III.C of [30] ) Mihailova gave a nice example of unsolvability of the 
membership problem. The construction goes as follows. 

Like before, start with an arbitrary finite presentation, H = (si,...,s n \ Ri, . . . ,R m ), and 
consider the subgroup A = {(x,y) € F n x F n \ x =h y} ^ F n x F n . It is straightforward to 
verify that A = ((1, R\), . . . , (1, R m ), (si, si), . . . , (s„, s n )) (and so it is finitely generated), and 
that MP(A,F n x F n ) is solvable if and only if WP(H) is solvable. 

By Higman-Neumann-Neumann embedding theorem, we can restrict our attention to 2-genera- 
ted groups (take n = 2 in the above paragraph). From all this, we deduce the following. 

Theorem 7.4 Let F be a finitely generated group such that F<i x F<i embeds in Aut(F) in such 
a way that the image intersects trivially with Stab*{v), for some v E F. Then, Aut(F) contains 
an orbit undecidable subgroup; in other words, there exist F -by- [fig. free] groups with unsolvable 
conjugacy problem. □ 

In the rest of the section, we shall use Theorem 17.41 to obtain explicit examples in the free 
abelian and free cases. 



7.1 The free abelian case 



It is well known that F2 embeds in GL2CZ) and so, F2 x F2 embeds in GL^Z). Hence, we can 
deduce the following result. 

Proposition 7.5 For n ^ 4, GL n (Z) contains finitely generated orbit undecidable subgroups. 



Proof. Consider the subgroup of GL>2^L) generated by P = ( \ 3,) and Q = (f {), which is free 
and freely generated by {P, Q} as discussed in the proof of Corollary 16.31 We claim that (P, Q) n 



Stab* ((1,0)) = ((121))- In fact, it is clear that Stab* ((1, 0)) = Stab ((1,0)) ={(i ± °)|neZ} 
(and we can forget the negative signum because we are interested in the intersection with {P, Q) ^ 



SL 2 (Z)). Now, the image of (J?) 
Corollary 



~ l X4, under ip is x 12 2 x\ 



12 



for notation). So, 

(P, Q) n Stab* ((1, 0)) = ker ip n Stab((l, 0)) = ((x f r 1 x 4 ) 12 



x\ 2 € D12 (see the proof of 
((12 i))- 



Choose now a (free) subgroup {P',Q') ^ (P,Q) intersecting trivially with the cyclic subgroup 
(( y2 1)) (this always exists in non-cyclic free groups). And, for n ^ 4, consider 



D 



p> 








Id 



Q' 








Id 



Id 








P' 



Id 








Of 



^ GL 4 (Z) ^ GL r 



which is clearly isomorphic to F2 x F2. By construction, B intersects trivially with the (conjugacy) 
stabilizer of v = (1, 0, 1, 0, . . . , 0) G Z n . Finally, using Mihailova's construction, find a finitely 
generated subgroup A ^ B with unsolvable MP(A, B). By Proposition 17.31 applied to F = Z n , A 
is a finitely generated orbit undecidable subgroup of Aut(Z n ) = GL n (Z). □ 
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Corollary 7.6 There exist Z 4 -by- [f.g. free] groups with unsolvable conjugacy problem. □ 



While the constructions are quite different, these groups are reminiscent of Miller's examples, 
but with a free abelian base group. To the best of our knowledge, they are the first known examples 
of [f.g. free abelian]-by- [f.g. free] groups with unsolvable conjugacy problem. As a side consequence 
of the previous reasoning, we also obtain the following corollary. 

Corollary 7.7 For n > 4, CIP(GL n (Z)) is unsolvable. 

Proof. As noted above, the proof of Proposition 16.131 works entirely for any dimension n for 
which CIP(GL n (Z)) is solvable (for example, n = 2). But Proposition 17.51 states the existence of 
finitely generated orbit undecidable subgroups of GL n (Z), for n ^ 4. Hence, CIP(GL n (Z)) must 
be unsolvable in this case. □ 

7.2 The free case 

In order to apply Theorem 17.41 to the free group F3 = (q, a, b \ ) of rank 3, we need to identify a 
copy of F2 x F2 inside Aut{F^). For every w G (a, b), consider the automorphisms w 0\: F% — ► F3, 
q 1— > wq, a 1— > a, b \— > b, and \0 W : F3 — > F3, q 1— > qw, a 1— > a, b 1— > b. Clearly, Wl 9\ W2 0i = WlW2 0i and 
i9 Wl \9 W2 = i9 W2W1 , which means that { w 9\ \ w G {a, b)} ~ F2 and { \9 W \ w G (a, b)} ~ F% p ~ F2. 
It is also clear that Wl 9± \9 W2 = Wl 9 W2 = \9 W2 Wl 9\ (with the natural definition for Wl 6 W2 ). So, we 
have an embedding F2 x F2 ~ F^ p x i 7 ^ 13 — > J 4ut(i ? 3) given by (1^1,^2) 1— > w -i9 W2 , whose image is 



In order to use Proposition ^, 31 let us consider the element u = qaqbq. We claim that Bf)Stab*(v) = 
{Id}. In fact, suppose W\,W2 6 (a, &) are such that (v) Wl 9 W2 = Wiqw2aw\qw2bwiqw2 is conjugate 
to i> = qaqbq in F3. Since both words have exactly three occurrences of q, they must agree up to 
cyclic reordering. That is, q{w2awi)q{w2bw{)q{w2Wi) equals either qaqbq, or qbq 2 a, or q 2 aqb. From 
this, one can straightforward deduce that w\ = W2 = 1 in all three cases. Thus, Wl 9 W2 = Id proving 
the claim. 

Now, let H = (a, b | R±, . . . , R12) be Borisov's example of a group with unsolvable word problem, 
embedded in a 2-generated group via Higman-Neumann-Neumann embedding (see above and [21|). 
By Mihailova result and Proposition 17.31 A = (±9^, . . . , i#r 12 , a -i6 a , 6-1 0&) ^ Aut(Fs) is orbit 
undecidable. Hence, by Theorem 13.31 the i ? 3-by-i ? i4 group determined by the automorphisms 

10Ru ••• , 1^12) a- 1 ^; 6-1^6 ^ ^^(-^3)) 



has unsolvable conjugacy problem. This is precisely Miller's group G(H) associated to H = {a,b | 
Ri, . . . , R12) (see the beginning of the present section). Thus, the argument just given provides an 
alternative proof of Miller's Theorem 17.11 

Corollary 7.8 For n ^ 3, CIP(Aut(F n )) is unsolvable. 

Proof. As noted above, the proof of Proposition 16.211 works entirely for any rank n for which 
CIP(Aut(F n )) is solvable. But, for n ^ 3, Aut{F n ) contains finitely generated orbit undecidable 
subgroups. Hence, CW(Aut(F n )) must be unsolvable in this case. □ 



B = ( a -i01, 10a, 10b) = {w L 0w 2 \ W\, W 2 G (a,&)} ^ Aut(F 3 ). 



G 



q,a,b,ti, . . . ,ti2,di,d 2 t~r l ati = a 

t'Hti = b 



d ± qd\ = a qa 
d^ 1 ad\ = a 
d- l bdi = b 
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8 Open problems 



Finally, we collect several questions suggested by the previous results. 

Question 1. Apart from finitely generated abelian, free, surface and polycyclic groups, and 
virtually all of them (see Theorem 14. 8p . find more examples of groups F with solvable twisted 
conjugacy problem. 

Commentary. As mentioned in Section U for every group F with solvable twisted conjugacy prob- 
lem, the study of orbit decidability /undecidability among subgroups of Aut(F) becomes interesting 
because it directly corresponds to solving the conjugacy problem for some extensions of F.n 

Question 2. Is the twisted conjugacy problem solvable for finitely generated hyperbolic groups ? 

Commentary. The first step into this direction is the solvability of the twisted conjugacy problem 
for finitely generated free groups, proven in [6j. However, there is no hope to extend that proof 
for hyperbolic groups because we do not have enough control on the automorphism group of an 
arbitrary hyperbolic group. □ 

Question 3. Let F be a group given by a finite presentation (X \ R), and suppose we are given a 
set of words {w\, . . . , w r } on X such that K = (w\, . . . , w r ) ^ F is a finite index subgroup. Does 
solvability of TCP(F) imply solvability of TCP(K) ? Is it true with the extra assumption that K 
is characteristic in F ? 

Commentary. The reverse implication is proved to be true in Proposition 14.71 (i), under the char- 
acteristic assumption for K. However, to go down from F to K we would have to consider the 
apparently more complicated problem of dealing with possible automorphism of K which do not 
extend to automorphisms of F. Maybe this is a strong enough reason to build a counterexam- 
ple. Note that the answer to the non-twisted version of the same question is no by a result of 
Collins-Miller (see [12]). □ 

Question 4. Let F be Z n or F n , and let A ^ B ^ Aut(F) be two subgroups given by finite sets 
of generators, such that A has finite index in B, and MP(A,B) is solvable. Is it true that orbit 
decidability for B ^ Aut(F) implies orbit decidability for A $J Aut(F) ? 

Commentary. The reverse implication is proven in Proposition 16. 41 With the first argument there, 
a finite list of left coset representatives of A in B can be computed, say B = f3\A U • • • U f3 m A. 
Then, given u,v € F, and assuming we got (3 £ B such that v ~ u(3, the set of all such /3's is 
(3 ■ Stab(v) ■ Inn(F). Since generators for Stab(v) ■ Inn(F) are computable (by straightforward 
matrix calculations in the case F = Z n , and by McCool's algorithm in the case F = F n ), it only 
remains to decide whether the intersection (3 ■ Stab(v) ■ Inn(F)(~)A is empty or not. This can be done 
in the case n = 2 because CIP(GL2(Z)) is solvable (see the proof of Proposition 16.21]) . However, 
this last part of the argument does not work in the other cases, because CIP (Aut(F)) is unsolvable 
for F = 7L n with n > 4 (see Corollary \7.7\i , and for F = F n with n > 3 (see Corollary 17.80 . 

As partial answers, note that Propositions 16.111 and 16.191 show that the answer is yes in the 
special cases where F = 7L n and B = GL n (Z), and where F = F n and B = Aut(F n ), respectively. 

We formulate the question for free and free abelian groups because, if one allows an arbitrary 
ambient F, then the answer is negative: consider Collins-Miller example of a finitely presented 
group G with an index two subgroup F ^ G such that CP(G) is solvable but CP(F) is unsolvable 
(see [12J); furthermore, G contains an element go G G of order two which acts on F as a non-inner 
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automorphism 7 9o € Aut(F). So, we have the short exact sequence l—*-F—*-G—*C2—*l. Since 
CP(G) is solvable, the action subgroup B = {Id,~/ go } ^ Aut(F) is orbit decidable; however, CP(F) 
is unsolvable meaning that the trivial subgroup A = {Id} ^ Aut(F) is orbit undecidable. □ 

Question 5. Is any virtually solvable subgroup of Aut(F n ) orbit decidable? 

Commentary. This is the analog of Proposition 16.91 in the free setting. However, it reduces to the 
same question for virtually free abelian subgroups. In fact, Bestvina-Feighn- Handel proved in [1] 
that every solvable subgroup of Out(F n ) contains a finitely generated free abelian subgroup of index 
at most 3 5n (additionally, it is also known that free abelian subgroups of Out(F n ) have rank at 
most 2n — 3). And the same if true for Aut(F n ) because one can easily embed Aut(F n ) in Out{F2 n ) 
by sending a € Aut{F n ) to the outer automorphism of Fm which acts as a on both the first half and 
the second half of the generating set. So, the situation is formally simpler than in Proposition 16.91 
but the argument there does not work here because we cannot use the trick about polycyclic 
groups. Apart from the possible finite index step, this question asks for a multidimensional version 
of Brinkmann's result (Theorem 16. 17|) . So, due to the complexity of the proof and solution for the 
cyclic case, it seems a quite difficult question. 

It is worth remarking that Bestvina-Feighn-Handel also proved in [2] a strong version of Tits 
alternative for Out(F n ): every subgroup of Out{F n ) is either virtually solvable (and hence virtually 
free abelian) or contains a non-abelian free group. Since the same is true for subgroups of Aut(F n ) 
via the above embedding, an affirmative answer to Question 5 would then force orbit undecidable 
subgroups of Aut{F n ) to contain non-abelian free subgroups, like in the abelian context. This would 
intuitively confirm that, again, orbit undecidability is intrinsically linked to free-like structures. □ 

Question 6. Is any finitely presented subgroup of Aut(F n ) orbit decidable? 

Commentary. This question contains Question 5, so it is even more difficult to be answered in the af- 
firmative. Note that orbit undecidable subgroups of the form A = ( iOr 1} . . . , i#r 12 , a -i#a, ^ 
Aut(F^) corresponding to Miller's examples (see subsection IT.2 j) are not a counterexample to this 
question because they are not finitely presented by Proposition B in Grunewald [20] (there, F x^F 
corresponds to our A $J B ~ F2 x F2 ^ Aut(F^), and H corresponds to Borisov's group with two 
generators and unsolvable word problem). Alternatively, A is not finitely presented because it is not 
the direct product of finite index subgroups of F2 = { w 6\ \ w € (a, b)} and F2 = { \6 W \ w G (a, b)} 
(see Short's description of finitely presented subgroups of F2 x F2 in |38j). □ 

Question 7. Are there more sources of orbit undecidability other than exploiting the unsolvability 
of membership problem for certain subgroups ? 

Commentary. In order to find new sources, one needs to relate orbit decidability with some other 
algorithmic problem, for which there are known unsolvable examples. □ 

Question 8. Is it true that every finitely generated subgroup of GL^iZ) is orbit decidable ? Or 
conversely, is it true that there exists a Z 3 -by-free group with unsolvable conjugacy problem ? In 
close relation with this, is CIP(GL^(7j)) solvable ? 

Commentary. Propositions 16.131 and 17.51 and Corollaries 16.31 and 17.71 show that the cases of di- 
mension 2 and dimension bigger than or equal to 4 behave oppositely with respect to these three 
questions (answers being yes, no, yes, and no, yes, no, respectively). For the case of dimension 
3, we point out that GL^(7j) is not virtually free, so the argument given in Proposition 16.131 does 
not work in this case. But, on the other hand, F2 x F2 does not embed in GL^iZ) either (in fact, 
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only very simple groups G satisfy GxG^ GL$(Z)), so the argument in Proposition 17.51 does not 
work in dimension 3 either (unless one can find other pairs of subgroups A ^ B ^ GL^(1j) with 
unsolvable MP (.A, B)). In the free context, the situation is easier, with the difference in behavior 
happening between rank two and rank three (see Propositions I6.2T1 and [73]) , 

Also, it is interesting to remark that this situation is very similar (and maybe related) to the 
coherence of linear groups: it is known that GLi(7L) is coherent, because it is virtually free, and 
that GL n (Z) is not coherent for n ^ 4, precisely because it contains F2 x F% (see for example [20] 
and |38j). The question is still open in dimension 3, where none of the previous arguments work.D 

Question 9. Is it true that 

(a) for every two finitely generated groups A and B (except for A = 1 and \B\ < 00), there exists 
a finitely presented group G such that Aut(G) simultaneously contains an orbit decidable 
subgroup isomorphic to A, and an orbit undecidable subgroup isomorphic to B ? 

(b) for every finitely generated group A, there exists a finitely presented group G such that 
Aut(G) contains an orbit decidable subgroup isomorphic to A ? 

(c) for every finitely generated infinite group B, there exists a finitely presented group G such 
that CP(G) is solvable, and Aut(G) contains an orbit undecidable subgroup isomorphic to B 

? 

Commentary. Question (b) asks for a positive decisional condition, and question (c) for the corre- 
sponding negative one, while question (a) asks whether they are compatible within the same group 
G. Formally, (b) and (c) are partial cases of (a) (note that CP(G) is equivalent to OD({/d})). 
However, an easy construction using the direct product shows that affirmative answers for (b) and 
(c) would imply an affirmative answer for (a) too. 

Before, note that A = 1 and \B\ < 00 is the only situation where the copy of A will necessarily 
be a finite index subgroup of the copy of B and so, Proposition 16.41 would then say that solvability 
for OT)(A) implies solvability for OT)(B). In all other cases, even if B has a finite index subgroup 
isomorphic to A, it is conceivable that Aut(G) could contain copies of A and B apart enough to 
each other to fulfill the requirements of question (a). 

Now, let G\ and G2 be two groups, let G = G\ x G2 be its direct product, and understand any 
subgroup of Aut{Gi) as a subgroup of Aut(G) acting trivially on the other coordinate. It is easy to 
see that if the orbit decidability for A ^ Aut(Gi), and the conjugacy problem for G2 are solvable, 
then the orbit decidability for A ^ Aut(G) is also solvable. And similarly, if B ^ Aut(G2) is orbit 
undecidable then so is B ^ Aut(G). Hence, answering question (a) in the affirmative reduces to 
answer in the affirmative questions (b) and (c). 

If A is finitely presented, has trivial center, and CP(A) is solvable, then we can take G = A, 
and the copy of A in Aut{G) given by conjugations is clearly orbit decidable. This answers (b) in 
the affirmative in this very particular case. 

Finally, let B be a finitely generated, recursively presented group. By Higman's embedding 
theorem (see |25j), B embeds in a finitely presented group B' ^ 1, which then embeds in G\ = B'*X,. 
Since G\ has trivial center, Aut{G\) contains a copy of B given by inner automorphisms, say 
B\ ^ Aut(Gi). Take now another finitely presented group G2 with unsolvable conjugacy problem, 
and consider their free product, G = G\*G2- Extend the morphisms in B\ to automorphisms of G 
acting trivially on G2', this way, we obtain B2 ^ Aut(G), again isomorphic to B. Now, two given 
elements u, v G G2 lie in the same (B2 -Inn(G))-orbit if and only if they are conjugate to each other 
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in Gi\ thus, solvability of OD(i?2) would imply solvability of CP(G ! 2). Hence, B ~ B2 ^ Aut{G) 
is orbit undecidable. But, unfortunately, this does not solve question (c) because, by construction, 
CP(G) is unsolvable, like CP(G 2 ). 

Additionally, note that the recursive presentability for B in the previous paragraph, is an extra 
condition also satisfied in the main source of orbit undecidability presented above. Namely, all 
orbit undecidable subgroups coming from Theorem 17.41 are of Mikhailova's type and so recursively 
presented (since they have solvable word problem) . At the time of writing we are not aware of any 
construction producing orbit undecidable subgroups which are not recursively presented. □ 
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